Emc Corporation

Emc IP Holding Company LLC

Stephen Richard Ives

Peng Yin

Rong Yu

Adi Ofer

Qun Fan

Seema G Pai

Michael Garvey

Ningdong Li

Ran Margalit

Dale R Elliott

Roii Raz

Barak Bejerano

Michael J Scharland

Dan Aharoni

Yechiel Yochai

James Marriot Guyer

Gilad Sade

Edward S Robins

Kevin Granlund

Daniel E Rabinovich

Arieh Don

Marik Marshak

Jun Li

Hui Wang

Andrew Chanler

Scott Rowlands

Ofer Michael

Orit Levin-Michael

Jiahui Wang

John F Madden, Jr

Evgeny Malkevich

John W Lefferts

Pei-Ching Hwang

Cynthia Burns

Hagit Brit-Artzi

Michael Dudnik

Gabi Hershkovitz

Assaf Natanzon

Alexandr Veprinsky

Helen S Raizen

David Lionel Black

Amnon Naamad

Patrick Brian Riordan

Kunxiu Gao

Jeffrey Wilson

F William French

Zvi Gabriel Benhanokh

Hongliang Tang

Amnon Izhar

Patrick J Weiler

Prakash Venkatanarayanan

Kevin Rodgers

Zhigang Liu

Christopher Monti

Daohong Wang

Li Lang

Krishnamoorthy Anantharaman

Timothy Johnson

Daniel J Rodrigues

Sanjay Narahari

William P Powers, Jr

James P McCarthy

Amit Kumar

Akash Agrawal

Manickavasasaham M Senghuden

Shen Liu

Thomas M De Lucia

Jeffrey R Berenson

Sethu N Madhavan

Flexibly expanding the storage capacity of a data storage system by adding a single physical storage device or any number of disk drives to an existing storage system without the need to reconfigure existing erasure encoding groups of the system. The physical storage devices of a data storage system may be divided into a plurality of slices, and each slice may be a member of an erasure encoding group. Physical storage devices that are added to the data storage system may be divided into same number of slices and/to slices of a same size, which then may be added to existing erasure encoding groups, utilized as spare slices or left idle until all of the slices are integrated into the data storage system.

Flexible expansion of data storage capacity

Techniques for performing configuration may include: determining an initial distribution of logical devices among disk adapters, wherein the initial distribution assigns each logical device a designated disk adapter as a primary active disk adapter servicing I/Os directed to the logical device, wherein the logical devices have storage provisioned from a RAID group including physical storage devices, wherein the disk adapters concurrently issue I/Os to each of the physical storage devices; detecting a configuration change; and responsive to detecting a configuration change, performing first processing. The first processing may include determining, in accordance with the configuration change, a redistribution of the logical devices among the disk adapters; and draining pending I/Os. The configuration change may include adding or removing a logical device provisioned from the RAID group. Performing a first operation on a storage device of the RAID group may include synchronous draining of I/Os of the storage device.

Techniques for multiple directors concurrently servicing I/Os directed to same physical storage devices

A method of distributed management of recovery of multi-controller NVMe drives includes detecting a path failure of a PCIe path from a first storage node to a first controller on the multi-controller NVMe drive, and initially attempting to correct the path failure using a controller level reset. If the controller level reset is unsuccessful, an alternative path to the controller is sought, and if that is unsuccessful a drive level reset operation is coordinated by all storage nodes with controllers executing on the NVMe drive. To coordinate reset of the NVMe drive, one storage node is elected master. Each node (both slave and master) initiates quiescing of IO operations on its respective controller, and after quiescing has completed, initiates shutdown of its respective controller. Once all controllers are shut down, the master initiates reset of the NVMe drive. Timeouts are used to constrain completion of the quiescing and shutdown operations.

System and method for managing recovery of multi-controller NVMe drives

A storage system in one embodiment comprises a host processor, and a solid-state storage device coupled to the host processor and comprising a non-volatile memory and a storage controller. The host processor is configured to initiate an error scanning operation on a designated portion of the non-volatile memory by directing at least one read command to the storage controller. The read command is configured to indicate to the storage controller that data read from the non-volatile memory responsive to the read command is not to be returned to the host processor. For example, the read command may illustratively comprise a Scatter Gather List (SGL) bit bucket descriptor indicating that the data read from the non-volatile memory responsive to the read command is to be discarded rather than returned to the host processor. The storage controller records any detected errors in a media error log and notifies the host processor of such errors.

Storage system with host-directed error scanning of solid-state storage devices

Techniques for processing I/O operations may include performing DMA (direct memory access) operations between a data storage system, one or more physical storage devices, and a hardware component that communicate over at least one bus using a DMA-based protocol, such as NVMe (Non-Volatile Memory Express). The hardware device may perform encryption and decryption processing of data that is, respectively, stored to, and read from, physical non-volatile storage. The hardware device may optionally perform other processing for other data-related operations such as any of data validation and integrity checking, data deduplication, data compression, and data decompression. When performing DMA transfers, multiple descriptors, such as SGLs (scatter gather lists) or PRPs (physical region pages), for multiple data portions having logically contiguous consecutive logical addresses may be combined into a single descriptor sent in a single DMA operation.

Data encryption and decryption techniques

Described are techniques for processing requests at a data storage system. A request is received from a client to perform an operation with respect to a first data portion stored on physical storage devices of the data storage system. The first data portion is exposed through a set of at least two data nodes each accessing a same copy of the first data portion stored on the physical storage devices. The request is received at a first of the data nodes of the set. The request is processed with respect to the first data portion using the same copy accessible to each of data nodes of the set. The physical storage devices may be configured in a RAID group and the data nodes, optionally along with a name node providing metadata, may be embedded in the data storage system and execute in a virtualized environment.

Distributed file system deployment on a data storage system

Each SSD in an array is divided into splits that are contiguous LBA ranges of equal capacity. The splits are managed as if they were sub-drives or logical drives and used in RAID groups or other forms of redundancy schemes. A processor external to the SSD collects pending defect data log from the SSDs and determines which splits are associated with reported defects. Failed splits are rebuilt in spare splits using RAID or another redundancy scheme. Repaired splits are returned to service as spare splits. Irreparable splits are marked as lost. In the case of SSD failure the data splits of the SSD are rebuilt to a spare SSD. If enough failed splits of the failed SSD can be repaired then the repaired SSD may be returned to service.

Resilience of data storage systems by managing partial failures of solid state drives

Selective use of a software path and hardware path help to provide fine-grained T10-PI support while maintaining IO operation efficiency for single IO read/write commands transferring multiple data segments. NVMe hardware capability (i.e. the hardware path) is always utilized for CPU-intensive CRC verification. NVMe hardware capability is utilized for application tag and reference tag verification whenever possible. Software running on a computing node (i.e. the software path) is used for application tag and reference tag verification and replacement when those functions cannot be implemented by the NVMe hardware.

Software-assisted fine-grained data protection for non-volatile memory storage devices

A distributed network of storage elements (DNSE) is provided in which the physical capacity of each drive is split into a set of equal sized logical splits which are individually protected within the DNSE using separate RAID groups. To reduce restoration latency, members of the RAID groups having a member in common on a given drive are spread within the DNSE to minimize the number of sets of drives within the DNSE that have RAID members in common. By causing the splits to be protected by RAID groups, restoration of the splits may occur in parallel involving multiple drives within the DNSE. By minimizing the overlap between RAID members on various drives, failure of a given drive will not require multiple reads from another drive in the DNSE. Likewise, spare splits are distributed to enable write recovery to be performed in parallel on multiple drives within the DNSE.

Allocating RAID storage volumes across a distributed network of storage elements

Described are techniques for processing a data operation in a data storage system. A front-end component of the data storage system receives the data operation. In response to receiving the data operation, the front-end component performs first processing. The first processing includes determining whether the data operation is a read operation requesting to read a data portion which results in a cache miss; and if said determining determines that the data operation is a read operation resulting in a cache miss, performing read miss processing. Read miss processing includes sequential stream recognition processing performed by the front-end component to determine whether the data portion is included in a sequential stream.

Techniques for data prefetching

Individual storage devices of a RAID group are monitored for faults. A health indicator for each storage device is calculated based on fault growth rate. Non-failed storage device are swapped out based on the health indicator. Techniques for monitoring the storage devices include background media scans and growth list polling.

RAID failure prevention

Described are techniques for performing data migration for a source logical volume and a target. The target is configured as storage for another mirror of the source logical volume prior to copying data from the source logical volume to the target, and if the target is configured storage of another logical volume of the data storage system, the configured storage is remapped as storage for another mirror of the source logical volume prior to copying data from the source logical volume to the target. One or more invalid bits are set indicating that the target does not contain a valid copy of data from the source logical volume. Data is copied from the first mirror of the source logical volume to the target. Invalid bits are cleared as data portions of the first mirror of the source logical volume are copied to the target.

Techniques for performing data migration

A system and techniques are provided for evaluating front end activity of a storage device in connection with storage tiering and management operations at the back end without the need to collect new sub-LUN metrics/statistics on the front end or querying the front end. The system avoids demotions of extents that are hot on the front end even when seemingly cool on the back end. The result is improved system performance since data that is hot on the front end will not be demoted to a lower storage tier. The system provides that there is little to no additional performance impact on the front end because there is no requirement to collect any new front end statistics or query the front end, and there is no increase in the meta data collected by the automated tiering system as there is no new sub-LUN metrics collected on the front end.

Non-intrusive automated storage tiering using information of front end storage activities

Described are techniques for processing data operations. A read request for first data is received at a data storage system. It is determined whether the read request results in a cache hit whereby the first data is stored in a cache of the data storage system, or whether the read request otherwise results in a cache miss. If the read request results in a cache miss, processing is performed to determine determining whether to perform cacheless read processing or deferred caching processing to service the read request. Determining whether to perform cacheless read processing or deferred caching processing is performed in accordance with criteria including a measurement indicating a level of busyness of a back-end component used to retrieve from physical storage any portion of the first data not currently stored in the cache.

Techniques using cacheless read and deferred caching

Growing community of inventors

West Boylston, MA, United States of America

Stephen Richard Ives