Dell Products Inc.

Emc IP Holding Company LLC

Dell & Products L.p.

Dror Zalstein

Vladimir Shveidel

Bar David

Jenny Derzhavetz

Uri Shabi

Ami Sabo

Ronen Gazit

Nimrod Shani

Bar Harel

Adi Katzengold

Dennis Rusakov

Dafna Levi-Yadgar

Logical address space portions and virtual layer blocks (VLBs) can be partitioned into multiple sets. Each of multiple nodes in a system can be assigned exclusive ownership of one of the multiple sets. In at least one embodiment, for a read I/O which is received at a first node and directed to a logical address LAthat is owned by a second node, the first node can request that the second owning node perform resolution processing for LA. The second node can return either a VLB address or a PLB address based on whether the second node owns a VLB used in mapping LAto a corresponding physical location PAwhich includes content Cstored at LA. The second node can set a flag in its response to indicate whether a returned address is a VLB address or a PLB address.

Read I/O processing techniques using remote mapping resolution with logical address space slicing

A searchable metadata change log is maintained in a persistent memory of a data storage system, and stores metadata changes to be aggregated and applied to data storage system metadata located in non-volatile data storage. New metadata changes are added to the metadata change log by storing the new metadata changes into the persistent memory, storing structural modifications to the metadata change log needed to add the new metadata changes to the metadata change log into a transaction entry located in the persistent memory, and processing the resulting transaction entry to perform the structural modifications to the metadata change log that were stored in the transaction entry.

Managing changes to metadata in a data storage system

In at least one embodiment, processing can include: receiving metadata updates for MD pages; recording the MD updates for the MD pages in a MD log; and performing a first destaging phase that writes the MD updates from the MD log to a tablet stored on non-volatile storage. The tablet can include a bucket area and an extension area. The bucket area can include buckets each of which is a predetermined bucket size. The first destaging phase can include: writing a portion of the MD updates from the MD log to the bucket area; and writing a second portion of the MD updates from the MD log to the extension area. Each MD update of the second portion can overflow a corresponding one of the buckets which is included in the bucket area and which is associated with one of the MD pages updated by the MD update.

Techniques using a variable size extension area with two phase destaging

In at least one embodiment, processing can include: receiving, at a first node, a read operation that reads content of a logical address, wherein a second node, but not the first node, owns the logical address; and performing optimized read processing for the read operation. The optimized read processing can include: performing, in parallel, first processing that obtains a first address hint and first content corresponding to the logical address, and second processing that obtains a second address hint corresponding to the logical address; determining whether the first and second address hints match; if the first and second address hints match, determining that first content is valid content stored at the target logical address; if the first and second address hints do not match, determining the first content is not stored at the logical address, and using the second address hint to obtain second content stored at the logical address.

Double-mapping technique of I/O latency optimization

In at least one embodiment, processing can include determining, by a first node, an update to a metadata (MD) page, wherein the first node includes a first cache; sending, from the first node to a second node, a commit message including the update to the MID page; receiving, at the second node, the commit message from the first node; and storing, by the second node, an updated version of the MID page in a second cache of the second node only if the second cache of the second node includes a cached copy of the MD page, wherein the updated version of the MID page, as stored in the second cache of the second node, is constructed by applying the first update to the cached copy of the first MD page.

Techniques for maintaining cache coherency

In a dual node system, each node can store metadata updates in a volatile memory metadata log. Metadata pages can be used to access data stored on a volume. A node designated as preferred or affined for a volume can be expected to receive I/Os directed to the volume. A preferred node for a volume can record, in its volatile memory metadata log, the metadata changes for the metadata pages used to access data stored on the volume. A non-preferred or non-affined node for the volume can infrequently receive I/Os directed to the volume. A non-preferred node for a volume can record, in its volatile memory metadata log, pointers or references to the metadata changes for the metadata pages used to access data stored on the volume, where the pointers or references map to persisted copies of the meta changes as stored in a persisted metadata log.

Techniques for recording metadata changes

A method, computer program product, and computer system for controlling, by a computing device, access to a non-volatile memory using a non-volatile lock as a reader of the non-volatile memory. Metadata (MD) non-volatile memory commits may be throttled until capacity of the non-volatile memory is at a threshold capacity.

System and method for non-disruptive upgrade for metadata log with minimal hiccup

In at least one embodiment, processing can include: receiving a request for a transaction of MD (metadata) updates including a first MD update of a first MD update type of a first set and including a second MD update of a second MD update type of a second set; storing, in a first volatile MD log, the first MD update; storing, in a second volatile MD log, the second MD update; storing, in a first non-volatile MD log, the first MD update; and storing, in a second non-volatile MD log, the second MD update, wherein each MD update of the first volatile MD log and the first non-volatile MD log has a corresponding MD update type of the first set, and each MD update of the second volatile MD log and the second non-volatile MD log has a corresponding MD update type of the second set.

Techniques for efficient journal space handling and recovery processing with multiple logs

A method, computer program product, and computing system for receiving a flush request for a metadata page stored in a storage array of a multi-node storage system. The flush request may be queued on a flush request lock queue on at least one node of the multi-node storage system. One or more flush requests may be processed, via multiple nodes of the multi-node storage system, on the metadata page based upon, at least in part, the flush request lock queue.

System and method for lockless destaging of metadata pages

A method, computer program product, and computing system for assigning a plurality of unique sequential identifiers to a plurality of tablets in a cache memory system. One or more metadata deltas associated with a metadata page stored in a storage array may be written to the plurality of tablets in the cache memory system. Each metadata delta stored in at least one tablet of the plurality of tablets may be written to the metadata page stored in the storage array, thus defining one or more destage tablets. A largest unique sequential identifier from the plurality of unique sequential identifiers assigned to the one or more destage tablets, may be written to the storage array, thus defining a current tablet identifier for the metadata page.

System and method for lockless reading of metadata pages

A method, computer program product, and computing system for receiving, at a node of a multi-node storage system, one or more updates to a reference count associated with a metadata block. One or more reference count deltas associated with the metadata block may be stored in a cache memory system of the node. An existing copy of the metadata block in a cache memory system of each other node of the multi-node storage system may be retained.

System and method for reducing reference count update contention in metadata blocks

A mapper tree for a logical volume is provided by storing, in each leaf node of the mapper tree, pointers to pages of non-volatile storage that store host data written to corresponding pages within a segment of the logical address space of the logical volume that corresponds to the leaf node. In response to receiving an initial write operation directed to a segment of the logical address space of the logical volume for which no leaf node currently exists in the mapper tree, a representation of a new leaf node is added to a super leaf node in the mapper tree that efficiently stores representations of multiple leaf nodes.

Mapper tree with super leaf nodes

Techniques for handling journal space in a storage cluster with multiple delta log instances. The techniques include writing delta updates for a respective metadata type to an 'active' set of data containers in a delta log instance and raw delta updates to a raw delta log, switching a designation of the 'active' set of data containers from &#x201c;active&#x201d; to &#x201c;de-staging&#x201d; once one or more of the &#x201c;active&#x201d; set of data containers has been filled, writing a bookmark for the respective metadata type to the raw delta log and a bookmark list, determining that a de-staging operation has been completed for writing the delta updates from the &#x201c;de-staging&#x201d; set of data containers to a storage array, determining that the bookmark for the respective metadata type is the oldest bookmark in the list, and reclaiming space between a tail of the raw delta log and the bookmark written to the raw delta log.

System and method of handling journal space in a storage cluster with multiple delta log instances

During a restart process in which metadata is loaded from at least one of a plurality of storage devices into a cache, a storage controller is configured to generate an IO thread in response to the receipt of an IO request, identify at least one metadata page of the metadata that is used to fulfill the IO request, and generate a loading thread in association with the received IO thread that is configured to cause the storage controller to perform prioritized loading of the identified at least one page of the metadata into the cache. The loading thread is detachable from the IO thread such that, in response to an expiration of the IO thread, the loading thread continues to cause the storage controller to perform the prioritized loading until the loading of the at least one page of the metadata into the cache is complete.

Growing community of inventors

Givatayim, Israel

Dror Zalstein