Emc IP Holding Company LLC

Alex Soukhman

Uri Shabi

Lior Kamran

Ronen Gazit

Vladimir Shveidel

Marcelo Taube

Zvi Schneider

David Krakov

Bar David

Maor Rahamim

Hillel Costeff

Alon Titelman

A technique for correcting errors in a data storage system operates while the data storage system remains online. The technique includes identifying an object for validation, scanning a plurality of pointers, and counting a number of pointers that point to the object. The technique further includes repairing a discrepancy between the count of pointers and a reference count stored in connection with the object.

Online error recovery

At least one processing device is configured to detect a failure event impacting at least a first storage node of a plurality of storage nodes of a distributed storage system, and responsive to the detected failure event, to modify an input-output (IO) shaping mechanism in each of the storage nodes in order to at least temporarily reduce a total number of IO operations that are concurrently processed in the distributed storage system. For example, modifying an IO shaping mechanism in each of the storage nodes illustratively comprises transitioning the IO shaping mechanism in each of the storage nodes from a first operating mode to a second operating mode that is different than the first operating mode. The second operating mode of the IO shaping mechanism illustratively has a relatively faster responsiveness to changes in IO operation latency as compared to the first operating mode of the IO shaping mechanism.

Dynamic modification of IO shaping mechanisms of multiple storage nodes in a distributed storage system

A method comprising: receiving, by a first storage node, an instruction to designate a first device file as the first storage node's primary device file for accessing a storage device designating, by the first storage node, the first device file as the first storage node's primary device file for accessing the storage device, the first device file being designated as the first storage node's primary device file for accessing the storage device in response to the first instruction; receiving, at the first storage node, an I/O command that is associated with the storage device; making a first attempt to complete the I/O command by using the first device file, detecting, by the first storage node, an error that is generated in response to the first attempt; designating, by the first storage node, a second device file as the first storage node's primary device file for accessing the storage device.

Recover time improvement mechanism after device path failure in a storage system

Techniques for caching may include: determining an update to a first data page of a first cache on a first node, wherein a second node includes a second cache and wherein the second cache includes a copy of the first data page; determining, in accordance with one or more criteria, whether to send the update from the first node to the second node; responsive to determining, in accordance with the one or more criteria, to send the update, sending the update from the first node to the second node; and responsive to determining not to send the update, sending an invalidate request from the first node to the second node, wherein the invalidate request instructs the second node to invalidate the copy of the first data page stored in the second cache of the second node.

Caching techniques

At least one processing device is configured to select a virtual block corresponding to a physical block in a storage system, wherein the virtual block comprises a plurality of pages with one or more pages of the virtual block respectively pointing to one or more data portions in the physical block. The at least one processing device is also configured to identify a target physical block and one or more virtual blocks corresponding to the target physical block. The at least one processing device is further configured to move at least one data portion from the physical block to the target physical block. Still further, the at least one processing device is configured to redirect the at least one page of the virtual block corresponding to the at least one moved data portion to one of the one or more virtual blocks corresponding to the target physical block.

Garbage collection in a storage system at sub-virtual block granularity level

A method, computer program product, and computing system for identifying one or more metadata pages stored in a storage array, thus defining a primary set of metadata pages. An alternative set of metadata pages may be generated from the primary set of metadata pages. A log of changes associated with the primary set of metadata pages may be generated. A copy of at least a portion of the primary set of metadata pages may be generated based upon, at least in part, the alternative set of metadata pages and the log of changes associated with the primary set of metadata pages. The alternative set of metadata pages may lag behind in time from the primary set of metadata pages by a predefined amount of time.

System and method for restoring metadata pages

A method for use in a storage system, the method comprising: receiving an I/O command; identifying a latency of a first storage device that is associated with the I/O command; and executing the I/O command at least in part based on the latency, wherein executing the I/O command based on the latency includes: performing a first action when the latency is less than a first threshold, and performing a second action when the latency is greater than the first threshold, wherein identifying the latency includes retrieving the latency from a latency database, and wherein the first storage device is part of a storage array, the storage array including one or more second storage devices in addition to the first storage device.

Media hiccup management in a storage system

A method for use in a storage system, comprising: detecting that at least one storage device in a redundant storage device array has failed, the redundant storage device array including a plurality of storage devices; storing, in a memory, a recovery cache containing recovered data blocks, the recovered data blocks containing data previously stored on the failed storage device that has been calculated as part of executing read commands before the redundant storage device array is rebuilt; receiving an I/O command that is associated with the redundant storage device array, the I/O command including one of a read command or a write command; and executing the I/O command by using the recovery cache, wherein the redundant storage device array includes a plurality of storage regions, wherein any of the storage regions includes a respective portion of each of the plurality of storage devices.

Method and apparatus for avoiding redundant data recovery

A method, computer program product, and computer system for identifying, by a computing device, a plurality of blocks. A maximum number of blocks of the plurality of blocks capable of being copied to a new block may be identified. Data from the maximum number of blocks of the plurality of blocks may be copied to the new block.

System and method for higher amortization with PLB and VLB defragmentation

An apparatus comprises at least one processing device configured, in conjunction with processing of an input-output (IO) operation by a first storage node of a distributed storage system, where processing of the IO operation requires access to at least one persistent storage device remote from the first storage node but local to a second storage node of the distributed storage system, to send a request from the first storage node to the second storage node identifying data associated with the operation. The processing device determines a buffer of the second storage node that is allocated for use by the second storage node for temporary storage of the data, and utilizes the buffer to provide the data to the second storage node to allow the second storage node to write the data, and/or to obtain the data from the second storage node after the second storage node has read the data.

Bandwidth efficient access to persistent storage in a distributed storage system

A technique for performing deduplication traverses a deduplication database and assigns digest values in the database to buckets, where each bucket covers a respective range of digest values. To deduplicate a particular candidate block, the technique generates a digest from the candidate block and searches for the computed digest in a subset of the buckets, where the subset is selected based on the computed digest. If a target block providing an exact match or a suitably close partial match is found in the subset of buckets, the technique effects storage of the candidate block at least in part by providing a reference to the target block.

Deduplicating full and partial block matches

A processing device maintains a data structure for a set of process flows executing on one or more processing cores of a storage system. The data structure comprises entries identifying particular execution instances of one or more persistence point functions, each modifying persistent state, with the execution instances having been designated as tested in conjunction with testing of one or more process flows of the set of process flows. The processing device detects an execution instance of a persistence point function of a given one of the process flows, determines whether or not the detected execution instance of the persistence point function has a corresponding entry in the data structure, and responsive to the detected execution instance of the persistence point function not having a corresponding entry in the data structure, initiates a test of the given process flow that includes the detected execution instance of the persistence point function.

Persistence points based coverage mechanism for flow testing in high-performance storage systems

A method, computer program product, and computing system for associating each data container of a first set of data containers in volatile memory with a metadata page. One or more metadata changes associated with a metadata page may be written to the data container associated with the metadata page. The one or more metadata changes stored in the first set of data containers may be written to a storage array.

System and method for aggregating metadata changes in a storage system

An aspect includes providing a metadata structure having a logical level that points to a virtual level and a physical level to which the virtual level points. The method also includes storing, at the virtual level, a reference counter for each of a plurality of virtual-level type storage address entries in the metadata structure, and providing a pointer in the metadata structure between each pair of a number of pairs of virtual level address entries in which corresponding pages share a set of common sectors. The reference counter tracks a number of instances in which a corresponding pointer points to a corresponding virtual level address entry. An aspect further includes storing a single instance of the common sectors at the physical level.

Metadata representation for enabling partial page duplication

Virtual block redirection clean-up is accomplished for a virtual block in a mapper tree for a logical volume that is redirected to a destination virtual block in the mapper tree. In response to redirection of the virtual block, a redirection notification is sent to each component in a set of dependent components. Each dependent component may store one or more pointers to pointers located within virtual blocks of the mapper tree. The redirected virtual block is cleaned up in response to determining that each dependent component has expressly indicated, in response to the redirection notification, that it does not store any pointer to any pointer that is located within the redirected virtual block.

Growing community of inventors

Raanana, Israel

Alex Soukhman