Emc IP Holding Company LLC

Dell Products Inc.

Itay Keller

Dan Aharoni

Michal Yarimi

Asaf Porath

Yonatan Shtarkman

Nir Milstein

Neta Peleg

Erez Webman

Dvir Koren

Rivka Mayraz Matosevich

Doron Tal

Ziv Dor

Zeev Shusterman

Tal Abir

Sanjay Narahari

Ron Stern

Avi Puder

Galina Tcharny

Jonathan Sahar

Benjamin Grimberg

Yoav Peled

Zohar Lapidot

Yuval Miron

An apparatus includes at least one processing device configured to identify a logical storage volume stored across multiple storage nodes of a distributed storage system, to obtain address range distribution information for the logical storage volume from at least a subset of the storage nodes, the address range distribution information indicating, for each of a plurality of distinct address ranges of the logical storage volume, which of the storage nodes locally stores data for that address range, and to generate, from the obtained address range distribution information, a mapping of the distinct address ranges to particular ones of the storage nodes, wherein the mapping is utilized to select paths for delivery of input-output operations to the storage nodes. Obtaining address range distribution information for the logical storage volume may comprise, for example, sending log page commands to each of the storage nodes to obtain asymmetric range access and/or template information.

Host path selection utilizing address range distribution obtained from storage nodes for distributed logical volume

Techniques are provided for implementing intelligent defragmentation in a storage system. A storage control system manages a logical address space of a storage volume. The logical address space is partitioned into a plurality of extents, wherein each extent comprises a contiguous block of logical addresses of the logical address space. The storage control system monitors input/output (I/O) operations for logical addresses associated with the extents, and estimates fragmentation levels of the extents based on metadata associated with the monitored I/O operations. The storage control system identifies one or more extents as candidates for defragmentation based at least on the estimated fragmentation levels of the extents.

Intelligent defragmentation in a storage system

Techniques are provided to recover from partial device errors of storage devices in a data storage system. A storage control system manages a storage device which comprises storage capacity that is logically partitioned into segments of equal size. The storage control system groups at least some of the segments of the storage device into a segment group. Each segment of the segment group is configured to store one or more data items and associated metadata items. The storage control system generates a parity data segment based on the segments of the segment group, and persistently stores the parity data segment in association with the segment group. In response to detecting a storage device error associated with a corrupted segment of the segment group, the storage control system utilizes the parity data segment and non-corrupted segments of the segment group to recover at least one missing data item of the corrupted segment.

Recovery from partial device error in data storage system

A storage control system is configured to obtain first data associated with a logical data device and to store the first data in a first entry of a log-structured array. The storage control system is further configured to invalidate a second entry of the log-structured array based at least in part on the storage of the first data in the first entry. The second entry comprises second data that was associated with the logical data device prior to obtaining the first data. The storage control system is further configured to determine that a first indication in a first metadata indicates that the invalidated second entry corresponds to a transaction log and to defer reclamation of the second entry based at least in part on the determination that the first indication in the first metadata indicates that the invalidated second entry corresponds to the transaction log.

Deferred reclamation of invalidated entries that are associated with a transaction log in a log-structured array

An apparatus comprises at least one processing device comprising a processor coupled to memory. The at least one processing device is configured to obtain an input-output request issued by an application executing on a compute node via at least one network and to identify a storage node as corresponding to the obtained input-output request based at least in part on the obtained input-output request. The at least one processing device is configured to associate information corresponding to the compute node with the input-output request and to submit the input-output request and the associated information that corresponds to the compute node to the storage node via the at least one network. The storage node is configured to submit a response to the input-output request to the compute node via the at least one network based at least in part on the information.

Direct response to IO request in storage system having an intermediary target apparatus

Techniques are provided for utilizing snapshot longevity information to manage snapshots in a snapshot-enabled storage system. A snapshot management system generates a snapshot volume tree which comprises snapshots associated with a given storage volume. The snapshot management system obtains a snapshot policy associated with the given storage volume, which comprises a snapshot creation and retention schedule that specifies retention lifespans of the snapshots. The snapshot policy is evaluated to automatically determine and assign respective longevity ranking values for the snapshots within the snapshot volume tree, wherein the longevity ranking value of a given snapshot represents a retention lifespan of the given snapshot as compared to retention lifespans of other snapshots as represented by their respective longevity ranking values. When a snapshot is deleted, the assigned longevity ranking values of the snapshots are utilized to select a snapshot to assume ownership of uniquely-written data that is owned by the deleted snapshot.

Assignment of longevity ranking values of storage volume snapshots based on snapshot policies

In some embodiments, a storage system comprises at least one processor coupled to memory. The processor is configured to obtain a write operation that comprises first data associated with a logical data device and to store the first data in a first entry of a log-structured array (LSA). The at least one processor is configured to invalidate a second entry based at least in part on the storage of the first data in the first entry. The second entry comprises second data associated with the logical data device that was stored in the second entry prior to obtaining the write operation. The at least one processor is configured to determine that a first indication in LSA metadata associated with the LSA indicates that the invalidated second entry comprises data that is awaiting replication and to defer reclamation of the second entry based at least in part on the determination.

Deferred reclamation of invalidated entries associated with replication in a log-structured array

An apparatus comprising a processing device is configured to communicate with a storage node of a storage system via at least one network and to communicate with a compute node that implements at least a portion of an application via the at least one network. The at least one processing device is configured to obtain an input-output request associated with the application from the compute node via the at least one network and to associate information corresponding to the compute node with the input-output request. The at least one processing device is further configured to submit the input-output request and the associated information that corresponds to the compute node to the storage node. The storage node is configured to submit a response to the input-output request to the compute node via the at least one network based at least in part on the information.

Direct response to IO request in storage system with remote replication

An apparatus in an illustrative embodiment comprises at least one processing device comprising a processor coupled to a memory. The apparatus is configured to maintain a snapshot tree data structure having a plurality of volume nodes corresponding to respective ones of (i) a root volume and (ii) multiple snapshots related directly or indirectly to the root volume. The apparatus is further configured to receive a request to read a data item from a given volume offset of a particular one of the volume nodes, to determine a set of data descriptors for the given volume offset, to determine a set of volume nodes of interest for the particular volume node, to determine a contribution set based at least in part on the set of data descriptors and the set of volume nodes of interest, to determine a read address for the data item as a function of the contribution set, and to read the data item from the read address.

Snapshot-enabled storage system implementing algorithm for efficient reading of data from stored snapshots

An apparatus in an illustrative embodiment comprises at least one processing device comprising a processor coupled to a memory. The apparatus is configured to maintain a snapshot tree data structure having a plurality of volume nodes corresponding to respective ones of (i) a root volume and (ii) multiple snapshots related directly or indirectly to the root volume. The apparatus is further configured to determine a set of data descriptors for a given volume offset, to determine a set of reader volume nodes that are readers of a corresponding data item based at least in part on the set of data descriptors, to adjust one or more of the data descriptors in the set of data descriptors based at least in part on the set of reader volume nodes, and to reclaim storage space previously allocated to the data item responsive to the adjusting of the one or more data descriptors.

Snapshot-enabled storage system implementing algorithm for efficient reclamation of snapshot storage space

Techniques are provided for utilizing snapshot longevity information to manage snapshots of storage volumes in a snapshot-enabled storage system. A snapshot management system maintains a snapshot volume tree which comprises a plurality of snapshots having respective longevity ranking values. The longevity ranking value of a given snapshot represents an expected life span of the given snapshot as compared to expected life spans of other snapshots as indicated by their respective longevity ranking values. When a snapshot is deleted, the snapshot management system determines a set of candidate snapshots which can assume ownership of uniquely-written data of the deleted snapshot, evaluates the longevity ranking values of the candidate snapshots to select a candidate snapshot to assume ownership of the uniquely-written data of the deleted snapshot, and modifies metadata of the selected snapshot to transfer the ownership of the uniquely-written data from the deleted snapshot to the selected snapshot.

Storage system implementing snapshot longevity ranking for efficient management of snapshots

Techniques are provided for managing snapshot volume trees in a snapshot-enabled data storage system. A snapshot management system is configured to maintain a snapshot volume tree for a storage volume of a storage system, wherein the snapshot volume tree comprises a data structure which comprises a plurality of snapshot volume nodes corresponding to respective ones of (i) a root volume and (ii) multiple snapshots related directly or indirectly to the root volume, wherein the snapshot volume nodes comprise respective generation times which denote times that the respective snapshot volume nodes were created. The generation times are used in conjunction with write times of data items that are written to the snapshot volume nodes to manage snapshot reading and deletion operations.

Management of volume snapshots in a data storage system

A storage control system receives an I/O request from a client for accessing storage resources that are logically divided into device groups, and determines a resource token request value associated with the I/O request and a target device group to which the I/O request is directed. The storage control system determines a number of allowed resource tokens to allocate to the client as a function of (i) the resource token request value, (ii) a sum total of resource tokens requested by other clients for accessing the target device group, and (iii) a total amount of resource tokens currently allocated to the target device group to which the I/O request is directed. The storage control system sends the determined number of allowed resource tokens to the client to thereby enable the client to limit a number of inflight I/O requests that the client issues to the storage control system.

Management of shared resources in a software-defined storage environment

Techniques are provided for managing a least recently used cache using a linked list with a reduced memory footprint. A cache manager receives an I/O request comprising a target address, wherein the cache manager manages a cache memory having a maximum allocated amount of cache entries, and a linked list having a maximum allocated amount of list elements which is less than the maximum allocated amount of cache entries. If the target address does correspond to a cache entry, the cache manager accesses the cache entry to obtain the cache data from cache memory, removes a list element from the linked list, which corresponds to the accessed cache entry, selects an existing cache entry which currently does not have a corresponding list element in the linked list, and adds a list element to a head position of the linked list which corresponds to the selected cache entry.

Managing least recently used cache using reduced memory footprint sequence container

Techniques are provided for managing storage device errors during processing of inflight input/output (I/O) operations. A storage control system (e.g., a software-defined storage system) receives an I/O write request which includes data to be written to at least one storage device of a plurality of storage devices managed by the storage control system, and commences a write operation to write the data to the at least one storage device. In response to a storage device I/O error resulting from a failure of the write operation associated with the at least one storage device, the storage control system accesses a logical storage device in a non-volatile system memory device, and writes the data to the logical storage device in the non-volatile system memory device to complete the failed write operation.

Growing community of inventors

Tel Aviv, Israel

Itay Keller