Emc IP Holding Company LLC

Emc Corporation

Huawei Technologies Co., Limited

Robert Foley

Peter Puhov

Marc C Cassano

Daniel E Cummins

Ronald D Proulx

David W Harvey

Socheavy D Heng

Ashok Tamilarasan

Jian Gao

Geng Han

Jibing Dong

Adi Ofer

Peter John McCann

Steven A Morley

Lili Chen

Naizhong Chiu

Wayne E Garrett, Jr

Henry Austin Spang, Iv

Zhiqi Liu

A method is used in managing storage operations in storage systems. Based on a set of criteria, an amount of storage resources required to perform a storage operation is determined. The storage operation is directed to fault tolerant storage devices. The amount of storage resources is allocated prior to start performing the storage operation. The storage operation is performed by using the allocated storage resources.

Managing storage operations in storage systems

A computer-implemented method for remote direct memory access (RDMA) by a distributed storage node includes receiving a request for an input/output (I/O) process associated with data. In response to the request, a memory segment shared between the operating system and a user process is allocated using an operating system driver of the node. The user process includes an I/O stack for processing the request. The shared memory segment includes a context memory portion storing context information associated with the I/O stack, a header memory portion storing header information for the I/O process, and a data memory portion for storing the data. The shared memory segment is registered for RDMA access with a target storage node. An RDMA transfer is performed between the shared memory segment and the target node to complete the I/O process. The shared memory segment is deallocated upon completion of the RDMA transfer.

Input/output processing in a distributed storage node with RDMA

Disclosed herein are techniques for use in managing data storage. For example, in one embodiment, the techniques comprise determining a size of the write request. The size of the write request equating to half or more non-parity data portions in a full stripe of data but less than all non-parity data portions in the full stripe. The techniques comprise performing a process based on the size that includes reading a parity and one or more missing non-parity data portions in the full stripe, verifying the coherency of the missing non-parity data portions by comparing a checksum of the parity and a checksum of the missing non-parity data portions, determining a new parity based on the new data and the missing non-parity data portions, and writing, in response to verifying the coherency of the missing non-parity data portions, the new data and the new parity to a RAID group without writing the one or more missing non-parity data portions to the RAID group, wherein the new data and the new parity are written to the RAID group with a checksum.

Managing data storage

A data storage system includes interface circuitry receiving data storage requests and sending corresponding responses to data storage clients, and data storage devices providing non-volatile storing and retrieval of client data in response to the data storage requests. Processing circuitry executes software-implemented datapath modules that realize datapath functions to process the data storage requests and responses, as well as service infrastructure modules to realize a service infrastructure to provide for regularized communications among the datapath modules, including Notify, Monitor, and StartIO communications. Each datapath module registers with the service infrastructure to establish links into corresponding sub-modules that handle the Notify, Monitor, and Start IO communications, and each datapath module invokes a respective sub-module of another datapath module by performing regularized Notify, Monitor, and Start IO functions of the service infrastructure with an identification of the target datapath module which is to receive the corresponding regularized communication.

Data storage system employing service infrastructure for functional modules

There is disclosed herein techniques for managing data storage. In one exemplary embodiment, the techniques comprise generating one or more sets of storage devices. Each set is configured to be mutually exclusive with respect to other sets of the one or more sets such that storage devices within the same set provide the basis for the formation of a RAID (Redundant Arrays of Independent Disks) stripe. The techniques further comprise forming a RAID stripe from respective storage extents associated with different storage devices of the same set. The techniques further comprise mapping a storage object to the RAID stripe. The techniques further comprise servicing host I/O (Input/Output) operations directed to the storage object by using the RAID stripe mapped thereto.

Method, apparatus and computer program product for managing data storage

A technique for managing RAID storage in a data storage system provides a mapping subsystem and a RAID subsystem and employs the mapping subsystem to direct repair operations on damaged RAID stripes. The mapping subsystem stores metadata that provides information about data stored in the RAID subsystem and locations of that data on RAID stripes. In response to detection of a damaged RAID stripe, the mapping subsystem selectively determines, based on the metadata, whether to perform repair operations or to avoid repair operations. As many repair operations can safely be avoided, the disclosed technique has the effect of reducing unnecessary processing in the data storage system. When the RAID subsystem includes flash drives, the technique also reduces write amplification, thus preserving flash drives which might otherwise be subject to premature wear.

Selective raid repair based on content mapping

Techniques for managing restriping of data across drives of a data storage system in a mapped-RAID environment. In response to addition of a drive to existing drives of the data storage system, all drive extents of the data storage system are mapped out in an effort to consume as much free space as possible across the drives. Having mapped out the drive extents, how many free extents remain on each drive is determined. If a quantity of free extents on a drive exceeds a predetermined threshold number, then that drive is logically marked as requiring restriping. Restriping of data is then performed for all logically marked drives. Once the restriping of the data is completed, a determination is made as to whether any further restriping of data across the drives of the data storage system is required, taking into account a relative wear and/or utilization of the respective physical drives.

Mapped raid restripe for improved drive utilization

A technique for managing RAID (Redundant Array of Independent Disks) storage includes maintaining active-stripe metadata that stores, for each of multiple stripes that have been written by a RAID system, an entry that identifies the stripe and identifies a respective configuration plan in effect a last time the stripe was written. In response to a disk drive failure, the technique further includes generating a set of new plans that specify disk drive elements to be used in place of damaged ones and performing a selective repair operation. The selective repair operation iterates over a set of entries in the active-stripe metadata, performs a comparison between the identified plan for each stripe and a current plan implemented by the RAID system for writing to that stripe, and repairs the stripe when the identified plan and the current plan are different.

Repairing raid systems at per-stripe granularity

There are disclosed computer-implemented methods, apparatus, and computer program products for managing data storage. In one embodiment, the computer-implemented method comprises the step of receiving new data to be written to storage. The method also comprises the step determining that the new data does not form a full stripe of data. The method also comprises reading missing non-parity data in the stripe of data. The method further comprises determining new parity based on the new data and the missing non-parity data. The method still further comprises writing the new data and the new parity to storage in a manner that does not require the missing non-parity data to be written to storage.

A method, computer program product, and computing system for receiving an IO request for a RAID array. An actual queue depth of the RAID array is compared to a maximum queue depth for the RAID array. An IO queue weight is determined for the IO request. The IO queue weight is revised based upon a degrade mode condition for the RAID array, thus defining a revised IO queue weight. The actual queue depth is incremented based upon the revised IO queue weight.

IO request system and method

A technique for maintaining RAID (redundant array of independent disks) configuration metadata across multiple SPs (storage processors) includes receiving a change request by a controller within a first SP, writing, by the first SP, a RAID configuration change described by the change request to a persistent intent log, and informing a second SP that the intent log has been written. The second SP, upon being informed of the write to the intent log, reads the RAID configuration change from the intent log and writes the RAID configuration change to a persistent configuration database. In this manner, the first SP and the second SP both receive the RAID configuration change and thus are both equipped to service reads and writes directed to a affected RAID storage.

Synchronizing RAID configuration changes across storage processors

A method is used in managing mapping metadata in storage systems. A request is received to access data organized on a storage device of a storage system. Checksum information is used for validating the data. The checksum information is stored on the storage system separate from a location at which the data has been stored.

Managing mapping metadata in storage systems

Techniques are disclosed for using virtual drives to provide RAID groups in a data storage system. A pool of virtual drives is generated by i) dividing each one of a set of physical drives into multiple regions of physically contiguous non-volatile data storage, and ii) allocating individual regions of physically contiguous non-volatile data storage to corresponding virtual drives contained in a pool of virtual drives. RAID groups are then formed, where each one of the RAID groups is made up of a set of virtual drives assigned to the RAID group from the pool of virtual drives. One or more storage objects is then mapped to each RAID group, and host I/O operations received by the data storage system and directed to the storage object mapped to the RAID group are serviced using the set of virtual drives assigned to the RAID group.

Virtual drive based RAID groups

A method is used in managing storage device mappings in storage systems. A set of mappings is created in a distributed storage system. The distributed storage system stores data in a redundant manner at more than one location on a set of storage devices by using a set of stripes. Each stripe of the set of stripes indicates a data slice stored on a storage device. Each mapping of the set of mappings identifies information regarding location of data on a storage device for a stripe. The mapping is stored on the set of storage devices.

Managing storage device mappings in storage systems

Described are techniques for avoiding deadlock. Each consumer is assigned a priority level denoting a number of resource allocation requests that have been currently granted to the consumer. The priority level is incremented for each resource allocation request granted to the consumer. A set of common resource pool(s) and a reserved resource pool are provided. A wait list of pending resource requests made by the consumers is maintained. If no allocated resources of the set of common pool(s) have been released for at least a threshold amount of time and there is at least one request on the wait list, first processing is performed which includes selecting, in accordance with one or more criteria, a first pending resource request from the wait list that was made by a first consumer, and assigning the first consumer exclusive use of the reserved pool for a lifetime of the first consumer.

Growing community of inventors

Clinton, MA, United States of America

Robert Foley