Emc IP Holding Company LLC

Dell Products Inc.

Emc Corporation

Steven A Morley

Vamsi K Vankamamidi

Daniel E Cummins

Amitai Alkalay

Philippe Armangau

Rasa Raghavan

Lior Kamran

Gajanan S Natu

Shuyu Lee

Ryan Gadsby

Joel J Young

Vladimir Shveidel

Walter A O'Brien, Iii

Uri Shabi

David W Harvey

Alan Lee Taylor

Thomas Earl Linnell

Sudhir Srinivasan

Peter Puhov

Peter John McCann

Richard P Ruef

Socheavy D Heng

Robert Foley

Michael C Brundage

Michael Paul Wagner

Walter O'Brien

Michael J Noeth

Amital Alkalay

Doran Tal

System-level garbage collection is performed in an SSD-based data storage system employing log-structured writing of data. Respective reclaim unit sizes of the storage devices are identified, wherein each reclaim unit is a unit of programming and erasure used by device-level garbage collection to generate erased reclaim units for allocation for physical data storage. The data is arranged across the storage devices in first and second data units, each first data unit being of a first size U equal to the product of R and a stripe width W, each second data unit including a respective set of first data units which occupy a maximum-size reclaim unit of a storage device among those storing the first data units. The system-level garbage collection is operated with second-data-unit granularity to regularly generate empty second data units for allocation in the log-structured writing of data, reducing write amplification and device wear.

Log-structured data storage system using flexible data placement for reduced write amplification and device wear

Techniques manage data within storage equipment having a Flexible Data Placement (FDP) drive. The techniques involve configuring the storage equipment to support storing data in reclaim units of the FDP drive according to an initial group of reclaim unit handles. The techniques further involve detecting that a number of currently available reclaim units of the FDP drive has fallen below a first predefined threshold. The techniques further involve, in response to detecting that the number of currently available reclaim units of the FDP drive has fallen below the first predefined threshold, reconfiguring the storage equipment to support storing data in reclaim units of the FDP drive according to a smaller group of reclaim unit handles. The smaller group of reclaim unit handles has fewer reclaim unit handles than the initial group of reclaim unit handles.

Managing reclaim unit handles to control access to a flexible data placement drive

A method, computer program product, and computing system for measuring a total storage device throughput value for a workload processed on a storage device within a storage array enclosure of a storage system. A maximum storage device throughput value may be determined for the workload. A storage device saturation value for the storage device may be determined based upon, at least in part, the total storage device throughput value for the workload and the maximum storage device throughput value for the workload. One or more IO requests may be processed on the storage device based upon, at least in part, the storage device saturation value determined for the storage device.

System and method for managing storage saturation in storage systems

A method, computer program product, and computing system for determining an amount of data moved within a plurality of write input/output (IO) log structures of a storage system during a monitoring interval. A utilization value for each write IO log structure may be determined during the monitoring interval. Over-provisioning resources are allocated from the storage system to the plurality of write IO log structures based upon, at least in part, the amount of data moved within each write IO log structure and the utilization value for each write IO log structure.

System and method for dynamic resource over-provisioning in log structures

A method, computer program product, and computing system for determining a storage device saturation value for a workload processed on a storage device within a storage array enclosure of a storage system. A storage controller saturation value may be determined for a workload processed on a storage controller within the storage array enclosure of the storage system. A storage array enclosure saturation value may be determined based upon, at least in part, the storage device saturation value and the storage controller saturation value. One or more IO requests may be processed on the storage device using the storage controller within the storage array enclosure based upon, at least in part, the storage array enclosure saturation value determined for the storage array enclosure.

Techniques for providing a virtual federation approach to increasing efficiency of processing circuitry utilization in storage nodes with a high number of cores. The techniques include, for each of two (2) physical nodes, logically partitioning a plurality of cores into a first domain of cores and a second domain of cores. The techniques include designating the first domain of cores of each physical node as belonging to a first virtual node. The techniques include designating the second domain of cores of each physical node as belonging to a second virtual node. The techniques include operating the first virtual nodes on the two (2) underlying physical nodes as a first virtual appliance, and operating the second virtual nodes on the two (2) underlying physical nodes as a second virtual appliance. In this way, scalability and speedup efficiency can be increased in a multi-core processing environment with a high number of cores.

Virtual federation approach for efficient CPU utilization in storage systems with high number of cores

A method, computer program product, and computing system for measuring a total storage controller throughput value for a workload processed on a storage controller within a storage array enclosure of a storage system. A maximum storage controller throughput value may be determined for the workload. A storage controller saturation value may be determined for the storage controller based upon, at least in part, the total storage controller throughput value for the workload and the maximum storage controller throughput value for the workload. One or more IO requests may be processed on one or more storage devices associated with the storage controller based upon, at least in part, the storage controller saturation value determined for the storage controller.

A technique for achieving data mobility between storage systems having different block sizes includes identifying a baseline size that is common to the block sizes of the storage systems. The technique further includes generating fingerprints of blocks based on extents having the common baseline size. Fingerprint-based data mobility is then achieved by comparing extent-based fingerprints, or combinations thereof, across different storage systems to identify matching data.

Fingerprint-based data mobility across systems with heterogenous block sizes

A method is provided for use in a storage processor, the method comprising: receiving a write request, the write request including a request to store user data in an array that includes a plurality of solid-state drives (SSD); executing the write request by: identifying metadata that is associated with the write request, and writing the user data and the metadata to different data streams that are opened on the plurality of SSDs; wherein writing the user data and the metadata to different data streams causes: (i) the user data to be stored in one or more first erase units of any of the plurality of SSDs, and (ii) the metadata to be stored in one or more second erase units of any of the plurality of SSDs, such that no part of the metadata is stored on any of the one or more first erase units, and no part of the user data is stored on any of the one or more second erase units.

Using multi-stream storage devices in log structured storage system

A method performed by a computing device, of configuring a new design of a new data storage system (DSS) having initial configuration parameters is provided. The new design includes an initial plurality of storage drives. The method includes (a) collecting operational information from a plurality of remote DSSs in operation, the operational information including numbers of writes of various write sizes received by respective remote DSSs of the plurality of remote DSSs over time; (b) modeling a number of drive writes per day (DWPD) of the initial plurality of storage drives of the new DSS based on the collected operational information from the plurality of remote DSSs and the initial configuration parameters; (c) comparing the modeled number of DWPD to a threshold value; and (d) in response to the modeled number of DWPD exceeding the threshold value, reconfiguring the new DSS with an updated design.

Configuring new storage systems based on write endurance

A technique provides preferential treatment to metadata over user data when protecting data in a mapped-RAID system. The technique involves generating metadata based on user data, storing the metadata in mirrored storage of the mapped-RAID system (e.g., 2-way mirrored storage), and storing the user data in non-mirrored storage of the mapped-RAID system (e.g., writing user data with parity in accordance with RAID 6). The mapped-RAID system provides reliability preference toward maintaining the metadata over maintaining the user data while the metadata and the user data are stored within the mapped-RAID system. Accordingly, the technique is able to concurrently provide high reliability and high storage efficiency.

Providing preferential treatment to metadata over user data

A method of tuning performance of a data storage system includes calculating an estimate of parallel fraction and speedup characteristic for a data storage application executed by the data storage system. The estimate is calculated using linear regression of values (1/N, 1/X) that are generated from trial runs of the data storage application processing a workload using respective different numbers N of CPU cores to obtain corresponding performance values X. The method further includes configuring the data storage system to execute the data storage application using a number of CPU cores based on the estimate of parallel fraction and speedup characteristic.

Performance tuning a data storage system based on quantified scalability

Host data stored in one or more source physical extents of non-volatile data storage is identified as valid and determined to be infrequently written by host I/O requests, and is therefore compressed to generate a highly compressed version of the valid host data. The highly compressed version is then stored into at least one target physical extent. The valid host data may be initially compressed before it is stored in the source physical extent(s), and may be re-compressed to generate the highly compressed version. If the valid host data is also infrequently read, it may be recompressed using larger blocks of host data than were used to perform the initial compression. The performance tier of the target physical extent may be different from (e.g. lower than) the performance tier of the source physical extent. The technology may be embodied in a background process such as a garbage collector.

Deep data-compression

A method, computer program product, and computer system for identifying a first node that has written a first page of a plurality of pages to be flushed. A second node that has written a second page of the plurality of pages to be flushed may be identified. It may be determined whether the first page of the plurality of pages is to be flushed by one of the first node and the second node and whether the second page of the plurality of pages is to be flushed by one of the first node and the second node based upon, at least in part, one or more factors. The first node may allocate the first page of the plurality of pages and the second page of the plurality of pages to be flushed in parallel by one of the first node and the second node based upon, at least in part, the one or more factors.

System and method for dual node parallel flush

There is disclosed a technique for use in optimizing write operations for flash devices. A system having a plurality of flash based solid state drives receives a write request to overwrite existing data stored on the solid state drives with new data. The write request data is formatted using a write granularity having a first size and the solid state drives are configured with a write granularity having a second size. Corresponding existing data is retrieved. The new data and the existing data are subdivided into multiple corresponding subunits where each subunit has a size equal to the second size. Each new data subunit is compared with each corresponding existing data subunit to identify which new data subunits include modified data. The new data subunits identified as having modified data are written to corresponding locations on the solid state drives.

Growing community of inventors

Mendon, MA, United States of America

Steven A Morley