Emc IP Holding Company LLC

Dell Products Inc.

Emc Corporation

Andrew Chanler

Arieh Don

Kevin M Tobin

Gabriel Benhanokh

Michael J Scharland

Gabi Benhanokh

David Shadmon

Philip Miloslavsky

Matthew David Ivester

Jeffrey Held

Stephen Richard Ives

John Creed

Dale R Elliott

Owen Martin

Felix Shvaiger

Michael Ferrari

Jeffrey Wilson

Hongliang Tang

Prakash Venkatanarayanan

Jerome Cartmell

Kaustubh S Sahasrabudhe

Edward S Robins

Michael Garvey

Kevin Granlund

Zhigang Liu

Krishnamoorthy Anantharaman

Mathew Pilozzi

Shen Liu

Gabriel BenHanokh

One or more aspects of the present disclosure relate to service level input/output scheduling to control central processing unit (CPU) utilization. Input/output (I/O) operations are processed with one or more of a first CPU pool and a second CPU pool of two or more CPU pools. The second CPU pool processes I/O operations that are determined to stall any of the CPU cores.

CPU utilization for service level I/O scheduling

A node mask of a global metadata structure entry representing an MD portion in GM may be updated when an entry of a local MD table representing the MD portion is removed in response to receiving an IO operation, without negatively impacting performance of the IO operation. An update process that is independent of the process executing the IO operation may update the node mask so that performance of the IO operation is not negatively affected. In response to the entry for the MD portion being removed from the local MD table, an entry may be added to a queue. The update process may include accessing the entries in the queue, and, for each entry, updating the node mask (e.g., clearing a bit representing the processing node) and removing the entry from the queue.

IO metadata management for directly connected host

In a storage system that implements metadata paging, the page free pool is replenished in the background to reduce foreground evictions and associated latency on page-in. A two-level page eviction controller with cascaded proportional, integral, derivative (PID) controllers optimizes the size of the free page pool and optimizes the rate at which pages are freed in the background. By optimizing these two parameters the page eviction controller dynamically maximizes used pages (minimizing free pages) to increase the metadata cache hit ratio. Optimizing the parameters also reduces the chances of foreground page evictions, thereby reducing IO latency, during both steady state and burst page-in requests.

Cascading PID controller for metadata page eviction

A storage array uses paged metadata. Each storage director has access to a plurality of object storage systems which describe locations of paged metadata in backing storage. Each object storage system includes different types of inodes which describe objects in backing storage. The object storage systems are used to locate and relocate metadata for loading into global memory, and creation and deletion of objects. An object storage system may be selected based on factors including ratio of different inode types, locality of object usage and anticipated object activity level.

Deleting an object from an object storage subsystem for managing paged metadata

Writes to one or more physical storage devices may be blocked after a certain storage consumption threshold (WBT) for each physical storage device. A WBT for certain designated physical storage devices may be applied in addition to, or as an alternative to, determining and applying a user-defined background task mode threshold (UBTT) for certain designated physical storage devices. In some embodiments, the WBT and UBTT for a physical storage device designated for spontaneous de-staging may be a same threshold value. Write blocking management may include, for each designated physical storage device, blocking any writes to the designated physical storage device after a WBT for the designated physical storage device has been reached, and restoring (e.g., unblocking) writes to the designated physical storage device after storage consumption on the physical storage device has been reduced to a storage consumption threshold (WRT) lower than the WBT.

Managing write access to data storage devices for spontaneous de-staging of cache

Snapshots may be managed on a data storage system including logical storage unit including data portions. For a first logical storage unit, a first snapshot pointer structure may be provided including entries, each entry corresponding to a physical storage location at which data is stored for a data portion of the first logical storage unit at a particular point in time. A first virtual snapshot lookup table may be provided for a first portion of the first logical storage unit, the first virtual snapshot lookup table including a plurality of entries, each entry corresponding to a respective data portion of the first logical storage unit and including a reference to a respective entry of the first snapshot pointer structure. The virtual lookup table may correspond to multiple snapshots of the first logical storage unit that have a same value for each data portion of the at least first portion.

Snapshot metadata deduplication

Techniques for obtaining metadata may include: receiving, by a director, an I/O operation directed to a target offset of a logical device, wherein the director is located on a board including a local page table used by components on the board; querying the local page table for a global memory address of first metadata for the target offset of the logical device; and responsive to the local page table not having the global memory address of the first metadata for the target offset of the logical device, using at least a first indirection layer to obtain the global memory address of the first metadata. The global memory may be a distributed global memory including memory segments from multiple different boards each including its own local page table. Compare and swap operations may be used to perform atomic operations to ensure synchronized access when updating the distributed global memory.

Distributed shared memory paging

A host system performs I/O processing using metadata for a storage system, where none or some of the metadata is stored on the host system. The host system may be coupled to the global memory of the storage system along a communication path that includes an internal switching fabric of the storage system and does not include a network located externally to the storage system. The host system may exchange communications over the communication path to access indirection layers on the storage system to determine a global memory address of metadata corresponding to an I/O operation. The host system may include a host metadata table of global memory addresses for metadata of logical locations logical devices. The host system may query the host metadata table for a global memory address of metadata corresponding to a logical location specified in an I/O operation.

Host access to storage system metadata

Providing global values may include configuring a global memory to include a global counter and configuring processing cores to have private caches each including two sets of buffers, an update toggle and a read toggle. A processing core having a first private cache may perform processing to read a current value for the global counter including determining the current value of the global counter as a mathematical sum of a local counter value and a local delta value from one of the two sets of buffers of the first private cache identified by the read toggle. The processing core may perform processing to modify the global counter by a first amount by updating the local delta value from a specified one of the two set of buffers of the first private cache identified by the update toggle.

Implementation of global counters using locally cached counters and delta values

Physical storage devices may be managed for spontaneous de-staging from a cache, for example, by determining a background task threshold (UBTT), and controlling one or more physical storage devices to use the UBTT instead of the VBTT in determining when to enter background task mode. The determined UBTT may be less than the VBTT, which may cause the physical storage device to enter into background task mode earlier, which means that background tasks like garbage collection and write leveling may be performed more frequently. An ability to turn on and turn off management of physical storage devices for spontaneous de-staging of a cache may be provided. It may be desirable to turn off physical storage device management for spontaneous de-staging, for example, during peak workload hours, to prevent physical storage devices from entering into background task mode more frequently during certain times.

Managing data storage devices for spontaneous de-staging of cache

Relocating paged metadata

Creating an object in an object storage subsystem for managing paged metadata

In a system in which a plurality of computing elements share a cache, each computing element owns a stripe of the cache. Each stripe contains cache objects that are accessible to all computing elements but managed only by the owning computing element. Each computing element maintains an LRU FIFO queue in local memory for the cache objects owned by that computing element. Each computing element also maintains a separate hash table in local memory for each other computing element. The hash tables indicate access to cache objects that are owned by those other computing elements. Each computing element updates its LRU FIFO queue when it accesses cache objects that it owns. The hash tables are periodically distributed by all computing elements via RDMA so that the LRU FIFO queues of all computing elements can be updated based on accesses to owned cache objects by other non-owner computing elements.

Phased based distributed LRU for shared cache systems

Described are techniques for processing I/O operations. A read operation is received to read first data from a first location. It is determined whether the read operation is a read miss and whether non-location metadata for the first location is stored in cache. Responsive to determining that the read operation is a read miss and that the non-location metadata for the first location is not stored in cache, first processing is performed that includes issuing concurrently a first read request to read the first data from physical storage and a second read request to read the non-location metadata for the first location from physical storage.

Growing community of inventors

Berlin, MA, United States of America

Andrew Chanler