Emc IP Holding Company LLC

Emc Corporation

Huawei Technologies Co., Limited

Lili Chen

Jian Gao

Xinlei Xu

Peter Puhov

Geng Han

Jibing Dong

Lifeng Yang

Hongpo Gao

Shaoqin Gong

Wayne Li

Zhiqi Liu

Liam Xiongcheng Li

Hongliang Tang

Shay Harel

Li Wan

Ruiyong Jia

Jianbin Kang

Changyu Feng

Ree Sun

Wayne E Garrett, Jr

Zhihao Tang

William C Davenport

Jamin Jianbin Kang

Richard P Ruef

Ashok Tamilarasan

Robert Foley

Ning Wu

Marc C Cassano

Qingyun Liu

Kimchi Mai

Mei-Chun Chiang

Liu Zhiqi

Huzefa Hakimi

A computer implemented method comprises accessing data stored in memory of the computer system, wherein the memory includes a faster access storage medium and a slower access storage medium; determining respective exponential moving averages of a number of access operations performed on respective memory locations of the memory; and transferring a portion of the data into the faster access storage medium and transferring a portion of the data into the slower access storage medium according to the determined exponential moving average.

I/O hotspot detection using exponential moving average

An interconnected computer system includes a Peripheral Component Interconnect Express (PCIe) fabric, a first computer system communicatively coupled to the PCIe fabric, a second computer system communicatively coupled to the PCIe fabric, and a shared single-access hardware resource coupled to the PCIe fabric. The first computer system includes a first processor and first memory coupled to the first processor configured to store a first flag indicating a desire of the first computer system to access the shared single-access hardware resource and a turn variable indicating which of the first computer system and the second computer system has access to the shared single-access hardware resource. The second computer system includes a second processor and second memory coupled to the second processor configured to store a second flag indicating a desire of the second computer system to access the shared single-access hardware resource.

Apparatus and method for locking PCIE network having non-transparent bridging

A clustered storage system includes a plurality of storage devices, each of which contributes a portion of its memory to form a global cache of the clustered storage system that is accessible by the plurality of storage devices. Cache metadata for accessing the global cache may be organized in a multi-layered structure. In one embodiment, multi-layered structure has a first layer first including a first address array, and the first address array include addresses pointing to a plurality of second address arrays in a second layer. Each second address array in the second layer includes addresses, each of which points to data that has been cached in the global cache.

Method for accessing caches in clustered storage system, involves determining and reading second address from second address array in accordance with request

Described are techniques for data caching. Dirty cached data of a first logical address may be stored in a secondary cache included in multiple caching layers. The multiple caching layers may also include a primary data cache. Two copies of the dirty cached data may be stored in two different cache pages on two different physical devices of the secondary cache. The two cache pages may be marked as dirty indicating that data stored in the cache pages of the secondary cache is a more recent copy of data stored at the first logical address than other data stored on first non-volatile physical storage. Clean cached data of a second logical address may also be stored in the secondary cache where a single copy of the clean cached data is stored in physical storage of the secondary cache.

Caching techniques duplicating dirty data in secondary cache

A method is performed by an extent pool manager running on a data storage device. It is configured to manage assignment of disk extents provided by a pool of drives to a set of mapped RAID extents. The method includes (a) receiving an indication that a particular drive has triggered an end-of-life (EOL) condition based on an error count of that drive and a standard sensitivity setting, (b) in response to receiving the indication, changing a sensitivity setting of other drives to be less sensitive than the standard sensitivity setting, and (c) remapping disk extents from the particular drive to the other drives of the pool while the other drives continue operation using the changed sensitivity setting. An apparatus, system, and computer program product for performing a similar method are also provided.

Adjustable error sensitivity for taking disks offline in a mapped RAID storage array

Techniques for allocating mapped RAID extents of a RAID group may include: determining a pool of N physical storage devices; selecting M physical storage portions, wherein each of the M physical storage portions is selected from a different one of the N physical storage devices of the pool; and allocating a first mapped RAID extent as the selected M physical storage portions. The first mapped RAID extent may denotes a stripe of the RAID group. Physical storage portions for each mapped RAID extent may be selected from the N physical storage devices with a goal of maintaining even distribution of the selected portions among the N physical storage devices. Such selection may use a neighborhood matrix and a subset of all possible combinations of M physical storage devices that may be selected from the N physical storage devices.

Extent selection with mapped raid

A concurrent rebuild list indicates RAID extents to be concurrently rebuilt in response to a physical drive failure. When rebuilding of a RAID extent in the list completes, a next RAID extent to add to the list is selected that has a lowest relatedness score in a candidate set of RAID extents. The relatedness score indicates an amount of limitation with regard to concurrently rebuilding the candidate RAID extent in combination with the RAID extents remaining in the concurrent rebuild list. The relatedness score may be a sum of a weighted write score indicating limits on concurrent write operations when rebuilding a candidate RAID extent in combination with the RAID extents remaining in the concurrent rebuild list, and a read score indicating limits on concurrent read operations when rebuilding the candidate RAID extent in combination with the RAID extents remaining in the concurrent rebuild list.

Reducing concurrency bottlenecks while rebuilding a failed drive in a data storage system

A method performed in a Mapped RAID system includes: (a) in response to a source disk (SD) reaching EOL status: (1) copying a first source disk extent (DE) (belonging to a first RAID extent) from the SD to a first spare DE on a first destination disk and (2) copying a second source DE from the SD to a second spare DE (belonging to a second RAID extent) on a second destination disk, (b) in response to completely copying the first source DE from the SD to the first spare DE, mapping the first spare DE to belong to the first RAID extent in place of the first source DE, and (c) in response to completely copying the second source DE from the SD to the second spare DE, mapping the second spare DE to belong to the second RAID extent in place of the second source DE.

Distributing mapped raid disk extents when proactively copying from an EOL disk

A data storage system creates, maintains and utilizes logical storage structures including (1) a pool of device extents on user devices, organized into data portions of mapped RAID groups each having a data portion and a RAID group metadata element having (i) a basic portion and (ii) a mapping portion mapping each set of device extents to logical RAID extents per RAID type, (2) a pool logical device (e.g., an internal mapped RAID) in the pool, storing the mapping portions of the metadata elements of the mapped RAID group, and (3) a system logical device on separate system devices, storing (i) the basic portions of the metadata elements of the RAID groups, and (ii) a pool metadata element including a pool mapping portion for the pool logical device.

Data storage system with multi-level, scalable metadata structure

Techniques detect non-uniform distributions of RAID extents across the storage drives of a data storage appliance and rebalance the allocation of disk extents of the storage drives amongst the RAID extents. The improved techniques prevent the distribution of RAID extents across the storage drives of a mapped RAID system from becoming overly non-uniform.

Disk extent rebalancing in mapped RAID storage arrays

A technique for managing metadata in a data storage system designates block pointers as either sources or copies, where sources contribute to reference counts of pointed-to structures but copies do not. The technique maintains parent-child relationships between parent BPSs (block pointer sets) and child BPSs, where each BPS includes an array of block pointers. Each child BPS is created as a copy of a parent BPS and has block pointers initially designated as copies. The technique performs a metadata-merge operation to merge the block pointers of the parent BPS into those of a child BPS by promoting attributes of block pointers in the child BPS from copy to source, avoiding any need to perform reference count updates on structures pointed to by promoted block pointers.

Merging mapping metadata to promote reference counting efficiency

A method is used in managing mapping metadata in storage systems. A request is received to access data organized on a storage device of a storage system. Checksum information is used for validating the data. The checksum information is stored on the storage system separate from a location at which the data has been stored.

Managing mapping metadata in storage systems

A technique processes input/output (I/O) requests from a set of host computers. The technique utilizes data storage equipment which includes a set of physical storage drives and storage processing circuitry which accesses the set of physical storage drives. The technique involves storing, by the storage processing circuitry, a set of storage drive objects in main memory. The set of storage drive objects includes a set of zeroed chunk tables. Each storage drive object (i) represents a physical storage drive and (ii) includes a zeroed chunk table which identifies storage chunks of that physical storage drive that have been initialized to zero. The technique further involves receiving, by the storage processing circuitry, I/O requests from the set of host computers, and performing, by the storage processing circuitry, I/O tasks in response to the I/O requests. The I/O tasks include zero on demand operations based on the set of zeroed chunk tables.

Zero on demand operations based on zeroed chunk tables of storage drive objects stored in main memory

Described herein are techniques for multi-cache data storage systems. The techniques comprise providing a volatile memory device to store data. The techniques also comprise providing a non-volatile storage device to store data. The techniques further comprise configuring a data storage system to form a multi-cache data storage system comprising a primary cache formed by the volatile memory device and the non-volatile storage device.

Method, apparatus and computer program product for use in managing multi-cache data storage systems

Techniques of protecting data involve storing a data structure in a data storage system that associates indices with respective keys. In this data structure, each index corresponds to a point in time when a respective key was put into effect. Each key may then be used to decrypt data stored in the data storage system that was encrypted during a time interval following the point in time when the respective key was put into effect, but before a later key was put into effect. During a key update process, a new key is put into effect and a new entry is created in the data structure that associates a new index with the new key.

Growing community of inventors

Hopkinton, MA, United States of America

Lili Chen