Emc IP Holding Company LLC

Emc Corporation

Huawei Technologies Co., Limited

Peter Puhov

Robert Foley

Lili Chen

Marc C Cassano

Shay Harel

Daniel E Cummins

David W Harvey

Ronald D Proulx

Wayne E Garrett, Jr

Zhiqi Liu

Jian Gao

William C Davenport

Ashok Tamilarasan

Socheavy D Heng

Geng Han

Jibing Dong

Gregory William Lazar

Richard P Ruef

Naizhong Chiu

Millard C Taylor, Iii

Hongpo Gao

Jianbin Kang

Sudhir Srinivasan

Peter John McCann

Steven A Morley

Thomas N Dibb

Joris Johannes Wils

Henry Austin Spang, Iv

Walter O'Brien

Wei He

Zhipeng Hu

Kimchi Mai

Eric Petsching

Naizhong Chiu Qui

Huzefa Hakimi

A method is used in managing storage operations in storage systems. Based on a set of criteria, an amount of storage resources required to perform a storage operation is determined. The storage operation is directed to fault tolerant storage devices. The amount of storage resources is allocated prior to start performing the storage operation. The storage operation is performed by using the allocated storage resources.

Managing storage operations in storage systems

A computer-implemented method for remote direct memory access (RDMA) by a distributed storage node includes receiving a request for an input/output (I/O) process associated with data. In response to the request, a memory segment shared between the operating system and a user process is allocated using an operating system driver of the node. The user process includes an I/O stack for processing the request. The shared memory segment includes a context memory portion storing context information associated with the I/O stack, a header memory portion storing header information for the I/O process, and a data memory portion for storing the data. The shared memory segment is registered for RDMA access with a target storage node. An RDMA transfer is performed between the shared memory segment and the target node to complete the I/O process. The shared memory segment is deallocated upon completion of the RDMA transfer.

Input/output processing in a distributed storage node with RDMA

Techniques for creating snapshots of data storage objects that can perform certain operations (e.g., flushing dirty data, setting up extent pointers, allocating block storage space, etc.) during background (or deferred) processing. The disclosed techniques employ one or more extent copy trackers that can be created during processing of a transaction, while I/O request from host computers are suspended. The extent copy trackers are configured to perform some or all of the certain operations in the background, after the transaction has been committed and/or the processing of the transaction has been completed. By performing such operations during background processing, a processing time required to complete the snapshot transaction is reduced, thereby reducing latency in the resumption of the I/O requests from the host computers.

Fast object snapshot via background processing

A data storage system includes interface circuitry receiving data storage requests and sending corresponding responses to data storage clients, and data storage devices providing non-volatile storing and retrieval of client data in response to the data storage requests. Processing circuitry executes software-implemented datapath modules that realize datapath functions to process the data storage requests and responses, as well as service infrastructure modules to realize a service infrastructure to provide for regularized communications among the datapath modules, including Notify, Monitor, and StartIO communications. Each datapath module registers with the service infrastructure to establish links into corresponding sub-modules that handle the Notify, Monitor, and Start IO communications, and each datapath module invokes a respective sub-module of another datapath module by performing regularized Notify, Monitor, and Start IO functions of the service infrastructure with an identification of the target datapath module which is to receive the corresponding regularized communication.

Data storage system employing service infrastructure for functional modules

A technique for managing RAID storage in a data storage system provides a mapping subsystem and a RAID subsystem and employs the mapping subsystem to direct repair operations on damaged RAID stripes. The mapping subsystem stores metadata that provides information about data stored in the RAID subsystem and locations of that data on RAID stripes. In response to detection of a damaged RAID stripe, the mapping subsystem selectively determines, based on the metadata, whether to perform repair operations or to avoid repair operations. As many repair operations can safely be avoided, the disclosed technique has the effect of reducing unnecessary processing in the data storage system. When the RAID subsystem includes flash drives, the technique also reduces write amplification, thus preserving flash drives which might otherwise be subject to premature wear.

Selective raid repair based on content mapping

Techniques for managing restriping of data across drives of a data storage system in a mapped-RAID environment. In response to addition of a drive to existing drives of the data storage system, all drive extents of the data storage system are mapped out in an effort to consume as much free space as possible across the drives. Having mapped out the drive extents, how many free extents remain on each drive is determined. If a quantity of free extents on a drive exceeds a predetermined threshold number, then that drive is logically marked as requiring restriping. Restriping of data is then performed for all logically marked drives. Once the restriping of the data is completed, a determination is made as to whether any further restriping of data across the drives of the data storage system is required, taking into account a relative wear and/or utilization of the respective physical drives.

Mapped raid restripe for improved drive utilization

A technique for managing RAID (Redundant Array of Independent Disks) storage includes maintaining active-stripe metadata that stores, for each of multiple stripes that have been written by a RAID system, an entry that identifies the stripe and identifies a respective configuration plan in effect a last time the stripe was written. In response to a disk drive failure, the technique further includes generating a set of new plans that specify disk drive elements to be used in place of damaged ones and performing a selective repair operation. The selective repair operation iterates over a set of entries in the active-stripe metadata, performs a comparison between the identified plan for each stripe and a current plan implemented by the RAID system for writing to that stripe, and repairs the stripe when the identified plan and the current plan are different.

Repairing raid systems at per-stripe granularity

A method, computer program product, and computing system for receiving an IO request for a RAID array. An actual queue depth of the RAID array is compared to a maximum queue depth for the RAID array. An IO queue weight is determined for the IO request. The IO queue weight is revised based upon a degrade mode condition for the RAID array, thus defining a revised IO queue weight. The actual queue depth is incremented based upon the revised IO queue weight.

IO request system and method

A mapping apparatus comprises a mapper that translates from an input key to an output key in one or more storage devices. A pre-mapper for processing update operations comprises a plurality of mapping tablets and an in-memory look-up filter to identify a given mapping table storing a given input key. The mapping tablets comprise at least one dynamic in-memory ingress tablet and a plurality of persisted frozen tablets. For a given update operation, a first entry is added to one dynamic in-memory ingress tablet comprising the input key for the given update operation and a corresponding output key where data for the given update operation is stored; and a second entry is added to the look-up filter comprising the input key of the first entry and an identifier of the dynamic in-memory ingress tablet storing the first entry for the given update operation. The dynamic in-memory ingress tablet is persisted as a persisted frozen tablet. A given storage element is cloned by making a copy of the given storage element and assigning a new internal identifier to the given storage element and the copy.

Storage element cloning in presence of data storage pre-mapper

A technique for managing metadata in a data storage system designates block pointers as either sources or copies, where sources contribute to reference counts of pointed-to structures but copies do not. The technique maintains parent-child relationships between parent BPSs (block pointer sets) and child BPSs, where each BPS includes an array of block pointers. Each child BPS is created as a copy of a parent BPS and has block pointers initially designated as copies. The technique performs a metadata-merge operation to merge the block pointers of the parent BPS into those of a child BPS by promoting attributes of block pointers in the child BPS from copy to source, avoiding any need to perform reference count updates on structures pointed to by promoted block pointers.

Merging mapping metadata to promote reference counting efficiency

A technique for maintaining RAID (redundant array of independent disks) configuration metadata across multiple SPs (storage processors) includes receiving a change request by a controller within a first SP, writing, by the first SP, a RAID configuration change described by the change request to a persistent intent log, and informing a second SP that the intent log has been written. The second SP, upon being informed of the write to the intent log, reads the RAID configuration change from the intent log and writes the RAID configuration change to a persistent configuration database. In this manner, the first SP and the second SP both receive the RAID configuration change and thus are both equipped to service reads and writes directed to a affected RAID storage.

Synchronizing RAID configuration changes across storage processors

A method is used in managing mapping metadata in storage systems. A request is received to access data organized on a storage device of a storage system. Checksum information is used for validating the data. The checksum information is stored on the storage system separate from a location at which the data has been stored.

Managing mapping metadata in storage systems

Techniques are disclosed for drive zeroing that create and maintain a memory resident zeroing map for each drive in a data storage system. The disclosed techniques create, for each drive in an array of non-volatile data storage devices, a zeroing map. The zeroing map for a drive includes multiple slots, each of which corresponds to a data storage area of the drive. Each slot in the zeroing map for a drive stores a zeroing status of the corresponding data storage area. The value of a slot in the zeroing map may indicate that the zeroing status of the corresponding data storage area is i) zeroed, indicating that the corresponding data storage area has previously been zeroed, or ii) unknown, indicating that the zeroing status of the corresponding data is unknown, i.e. the corresponding data storage area may or may not have previously been zeroed.

Per-drive memory resident zeroing maps for drive zeroing in a data storage system

Techniques are disclosed for using virtual drives to provide RAID groups in a data storage system. A pool of virtual drives is generated by i) dividing each one of a set of physical drives into multiple regions of physically contiguous non-volatile data storage, and ii) allocating individual regions of physically contiguous non-volatile data storage to corresponding virtual drives contained in a pool of virtual drives. RAID groups are then formed, where each one of the RAID groups is made up of a set of virtual drives assigned to the RAID group from the pool of virtual drives. One or more storage objects is then mapped to each RAID group, and host I/O operations received by the data storage system and directed to the storage object mapped to the RAID group are serviced using the set of virtual drives assigned to the RAID group.

Virtual drive based RAID groups

A technique processes input/output (I/O) requests from a set of host computers. The technique utilizes data storage equipment which includes a set of physical storage drives and storage processing circuitry which accesses the set of physical storage drives. The technique involves storing, by the storage processing circuitry, a set of storage drive objects in main memory. The set of storage drive objects includes a set of zeroed chunk tables. Each storage drive object (i) represents a physical storage drive and (ii) includes a zeroed chunk table which identifies storage chunks of that physical storage drive that have been initialized to zero. The technique further involves receiving, by the storage processing circuitry, I/O requests from the set of host computers, and performing, by the storage processing circuitry, I/O tasks in response to the I/O requests. The I/O tasks include zero on demand operations based on the set of zeroed chunk tables.

Growing community of inventors

Shrewsbury, MA, United States of America

Peter Puhov