Emc IP Holding Company LLC

Dell Products Inc.

Nimrod Shani

Leron Fliess

Vladimir Shveidel

Vamsi K Vankamamidi

Tal Zohar

Lior Kamran

Ronen Gazit

Dror Zalstein

Jenny Derzhavetz

Anton Kucherov

Uri Shabi

Socheavy D Heng

Shari Vietry

Michael Litvak

Vikram Prabhakar

Amir Kredi

Nadav Krispin

A technique of managing metadata includes receiving metadata changes into active tablets. When the active tablets are filled, the active tablets are changed to frozen tablets, whereupon previously frozen tablets are freed and changed to the active tablets. The technique further includes destaging metadata changes from the frozen tablets to pages in backend storage under read locks. Tablet generation identifiers (TGIDs) are assigned to successive generations of tablets, and a TGID of the frozen tablets is stored with the pages that are written as part of destaging from the frozen tablets.

Destaging metadata changes using read locks

A technique for copying data from a source range to a destination range includes identifying a chunk of the source range. The chunk includes multiple logical blocks that map, through a series of source-mapping pages, to a set of virtual pages that point to physical data of the chunk. The technique further includes copying the series of source-mapping pages to form a corresponding series of destination-mapping pages pointed to by the destination range, so that a corresponding chunk of the destination range maps, via the series of destination-mapping pages, to the same set of virtual pages mapped to by the series of source mapping pages.

Metadata-based data copying

A technique for managing metadata changes in a storage system provides different aggregation policies for use with different metadata. The technique includes assigning metadata changes to respective aggregation policies and storing the assigned metadata changes in a set of data structures. The technique further includes destaging the metadata changes from the set of data structures separately for the different aggregation policies in accordance with settings specific to those aggregation policies.

Processing metadata changes using multiple aggregation policies

In at least one embodiment, processing can include: receiving, at a first node, a read operation that reads content of a logical address, wherein a second node, but not the first node, owns the logical address; and performing optimized read processing for the read operation. The optimized read processing can include: performing, in parallel, first processing that obtains a first address hint and first content corresponding to the logical address, and second processing that obtains a second address hint corresponding to the logical address; determining whether the first and second address hints match; if the first and second address hints match, determining that first content is valid content stored at the target logical address; if the first and second address hints do not match, determining the first content is not stored at the logical address, and using the second address hint to obtain second content stored at the logical address.

Double-mapping technique of I/O latency optimization

A method, computer program product, and computing system for dividing a logical address space into a first and at least a second set of data portions. Exclusive ownership of the first set may be assigned to a first storage node of a storage cluster. Exclusive ownership of the second set may be assigned to at least a second storage node of the storage cluster. One or more IO requests associated with one or more of the first set of data portions and the at least a second set of data portions may be processed without transferring data portions between the first the at least a second storage node based upon the exclusive ownership of the first set of data portions by the first storage node and the exclusive ownership of the at least a second set of data portions by the at least a second storage node.

System and method for managing data portion ownership in a storage cluster

A technique for managing a log in a storage system includes adding descriptors to the log, the descriptors indicating changes in user data that affects metadata, and creating a working set of descriptors that includes both per-block descriptors for specifying per-block changes and per-extent descriptors for specifying per-extent changes, where an extent includes a range of contiguous blocks. The technique further includes flushing the working set in a single flush cycle, including flushing the per-block descriptors together with the per-extent descriptors.

Combining xcopy's, unmaps, and writes in a single flush

A method, computer program product, and computing system for receiving data for storage in a storage system. The data may be written to a head entry of a log buffer, wherein the log buffer includes a plurality of data entries for flushing to the storage system. At least a portion of the plurality of data entries of the log buffer may be flushed, via a plurality of threads, to the storage system based upon, at least in part, a tail entry of the log buffer. A queue of committed data entries may be updated, via each thread of the plurality of threads, with one or more data entries of the log buffer flushed to the storage system by each thread. A new tail entry of the log buffer may be determined, via a thread of the plurality of threads, based upon, at least in part, the queue of committed data entries.

System and method for cooperative tail promotion in a log buffer

A method, computer program product, and computer system for receiving an XCopy command is provided. The XCopy command may be in the form of an IO operation. The IO operation may be a subextent block operation. A source range and a destination range of the XCopy command may be determined to be aligned within an alignment boundary. The Xcopy command may be determined to be smaller than a predetermined size. In response to determining the source range and destination range of the XCopy command are aligned within the alignment boundary and the XCopy command is smaller than a predetermined size, the XCopy command may be processed. The receiving of the XCopy command may be recorded in a log.

System and method for efficient implementation of XCopy commands

A method, computer program product, and computing system for identifying one or more first layer metadata blocks with a plurality of entries mapping to a plurality of entries of one or more second layer metadata blocks. The one or more second layer metadata blocks mapped to by the one or more first layer metadata blocks may be identified. A common back pointer for at least a subset of the plurality of entries of the one or more second layer metadata blocks may be generated, the common back pointer pointing from the at least a subset of the plurality of entries of the one or more second layer metadata blocks to the one or more first layer metadata blocks.

System and method for generating common metadata pointers

A method is disclosed for recovering metadata, the method comprising: identifying a corrupt or missing metadata entry, the corrupt or missing metadata entry being part of a mapping structure that is stored in a random-access memory (RAM) of a storage system; selecting a metadata backup page that is associated with the mapping structure; identifying a plurality of copies of the selected metadata backup page; identifying a most recent copy among the plurality of copies of the selected metadata backup page; and recovering the corrupt or missing metadata entry based on the most recent one of the plurality of copies of the metadata backup page.

Method to recover metadata in a content aware storage system

A method is provided for storing log data in a logging system, comprising: instantiating a plurality of buffers in the logging system, the plurality of buffers including one or more type-1 buffers and one or more type-2 buffers; retrieving a log data item from a temporary storage of the logging system, and identifying a type of the log data item; when the log data item is from a first type: identifying a current type-1 buffer, detecting whether the current type-1 buffer is full, when the current type-1 buffer is not full, storing the log data item in the current type-1 buffer, when the current type-1 buffer is full, designating another buffer as the current type-1 buffer, and storing the log data item in the newly-designated current type-1 buffer.

Logging infrastructure with time sustain requirements

An apparatus comprises at least one processing device comprising a processor coupled to a memory that is configured to initiate a read data request utilizing a logical address of a content addressable storage system that maps to a physical address comprising an offset on a storage device that internally maps the offset to a first sector. The processing device is also configured to determine a health of the first sector responsive to the read data request failing, to recover data stored in the first sector responsive to the first sector being a bad sector, and to overwrite the recovered data to the logical address while maintaining the mapping to the physical address by directing a write of the recovered data to the offset to update the internal mapping of the offset in the storage device to a new physical location corresponding to a second sector different than the first sector.

Maintaining logical to physical address mapping during in place sector rebuild

A technique handles metadata corruption to avoid data unavailability. The technique involves performing metadata evaluation operations on metadata describing pages of written data in a data-log that holds data en route to volumes in secondary storage. The technique further involves, while results of the metadata evaluation operations indicate that there is no corrupt metadata, flushing the pages of written data from the data-log to the volumes in the secondary storage. The technique further involves, in response to a result of a particular metadata evaluation operation indicating that metadata for a particular page of written data in the data-log is corrupt, quarantining the particular page of written data from the data-log to a containment cache to enable further flushing of other pages of written data from the data-log to the volumes in the secondary storage.

Handling metadata corruption to avoid data unavailability

A method is disclosed for recovering data, the method comprising: retrieving a mapping structure associated with a volume in a storage system, the mapping structure including a plurality of entries, each entry including a respective short hash digest of a data block stored in the volume and an identifier of a location where the data block is stored; generating a plurality of long hash digests, each of the long hash digests being generated based on a different one of a plurality of data blocks that are stored in the storage system; comparing each of the long hash digests to a short hash digest of a missing data block, and adding the long hash digest to a candidate list when the long hash digest matches the short hash digest of the missing data block; generating a filtered candidate based on the candidate list; and recovering the missing data.

User data recovery in a content aware storage system

A storage system in one embodiment comprises a plurality of storage devices and a storage controller. The storage controller is configured to scan data pages and associated metadata structures of the storage system, the metadata structures comprising at least a first metadata structure associating unique hash identifiers with physical locations of respective data pages in the storage system and a second metadata structure associating the physical locations of respective data pages with hash digests of the data pages and the unique hash identifiers. The storage controller is further configured to detect an error in one of the first and second metadata structures. Responsive to the detected error, the storage controller recovers a corresponding portion of the metadata based at least in part on one or more entries of the other one of the first and second metadata structures. The unique hash identifiers illustratively comprise respective shortened representations of the hash digests, such as hash handles uniquely corresponding to respective ones of the hash digests.

Growing community of inventors

Raanana, Israel

Nimrod Shani