Emc IP Holding Company LLC

Nexenta Systems, Inc.

Amc IP Holding Company LLC

Ilya Usvyatsky

Jian Gao

Geng Han

Jibing Dong

Hongpo Gao

Jamin Jianbin Kang

Xinlei Xu

Yousheng Liu

Shaoqin Gong

Nickolay Alexandrovich Dalmatov

Weihua Li

Lifeng Yang

Caitlin Bestler

Ree Sun

Wayne Li

Dmitry Yusupov

A technique for use in managing mapped RAID extents is disclosed. A neighborhood matrix denoting a distribution of neighboring physical storage portions across the N physical storage devices of the pool is created and initialized. A disk extent consumed array denoting a disk consumption metric is created and initialized. Disks in an extent pool are analyzed to identify a least consumed disk in the extent pool and a disk extent is allocated from the identified disk. The allocated disk extent is configured as a first disk extent for a first RAID extent. The neighborhood matrix and disk extent consumed array are updated appropriately. All the disks in the extent pool are analyzed to identify a valid disk extent that will result in the lowest variance value in the neighborhood matrix. The identified valid disk extent is allocated and the neighborhood matrix and disk extent consumed array are again updated.

Managing mapped raid extents in data storage systems

A technique accesses a non-resident segment and a resident segment of a segmented de-duplication index, the resident segment being currently loaded into primary memory from secondary storage for data block de-duplication, and the non-resident segment not being currently loaded into the primary memory from the secondary storage for de-duplication. The technique further discovers that a digest of a non-resident digest entry of the non-resident segment and a digest of a resident digest entry of the resident segment are duplicates. The non-resident digest entry includes a first reference to a first location of the secondary storage that holds a first data block copy, and the resident digest entry includes a second reference to a second location of the secondary storage that holds a second data block copy. The technique further performs reconciliation that conforms the non-resident segment and the resident segment of the index to reference only data block copy.

Performing reconciliation on a segmented de-duplication index

A method, computer program product, and computer system for reducing, by a computing device, overlap in a RAID extent group by moving a first disk extent in an extent pool from a source disk to a target disk in a Mapped RAID group. A neighborhood matrix for the Mapped RAID group may be balanced by moving a second disk extent in the extent pool from the source disk to the target disk in the Mapped RAID group.

Reducing overlap through shuffle and keeping relationship balance on mapped RAID system and method

In response to detecting that a total number of physical data storage drives in a group of physical data storage drives exceeds a maximum drive group size, the group of physical data storage drives is divided into a multiple partnership groups, and RAID extents in a RAID mapping table are divided into multiple RAID extent groups. Each one of the RAID extent groups contains multiple RAID extents and corresponds to one of the partnership groups of physical data storage drives. The RAID extents in each RAID extent group only indicate physical data storage drives contained in the corresponding partnership group of physical data storage drives, in order to ensure that data recovery and RAID extent rebuilding after a drive failure involve only the physical data storage drives within the same partnership group as the failed physical data storage drive.

Splitting a group of physical data storage drives into partnership groups to limit the risk of data loss during drive rebuilds in a mapped RAID (redundant array of independent disks) data storage system

Techniques have been provided for storing data in a de-duplicated manner on a data storage system in a manner that allows for real-time reference to an index that is too large to fit within memory. This may be accomplished by segmenting the index into smaller segments, stored on disk. Only a subset of the segments may be loaded into memory at a given time. A predictive filter is stored in memory for each segment, allowing a de-duplication driver to quickly predict whether any given new block is likely to be indexed by each segment. Since identical blocks are often stored in long identical sequences (e.g., upon copying a disk image to a disk for a virtual machine), once a segment stored on disk is referenced many times in a short period, it is loaded into memory to allow the remainder of the long sequence to be de-duplicated.

Techniques for de-duplicating data storage systems using a segmented index

A method, computer program product, and computer system for adding, by a computing device, a plurality of disks in an extent pool to an available disk list. It may be determined, for a Mapped RAID group, that each RAID extent in a sub group has not been allocated. At least one RAID extent in the sub group may be allocated based upon, at least in part, determining that each RAID extent in a sub group has not been allocated.

Avoiding storage device overlap in raid extent sub group and keeping relationship balance on mapped raid system and method

A method, computer program product, and computer system for initializing, on a computing device, an N*N Weighted Neighborhood Matrix according to an existing Mapped RAID layout. N disks in an original disk partner group (DPG) may be partitioned into a target DPG, where N is a number of disks in an extent pool, wherein the N disks have at least two different capacities. The target DPG may be marked for at least a portion of the N disks. The target DPG may be marked for E RAID extents, where E is a number of RAID extents in the original DPG. At least a portion of disk extents for the E RAID extents in the original DPG may be moved to the target DPG.

Storage device group split technique for extent pool with hybrid capacity storage devices system and method

A method, computer program product, and computer system for selecting, by a computing device, a first disk extent for each RAID extent in an extent pool. Remaining disk extents for each RAID extent in the extent pool may be selected.

Dynamic programming based extent selection for hybrid capacity extent pool system and method

Technology for synchronizing event handling between a primary storage processor and a secondary storage processor in an active/active data storage system. A single working task descriptor is selected at a time for uninterruptible execution, from a task scheduling queue that is maintained only on the primary storage processor. A copy of the working task descriptor is transmitted to the secondary storage processor. The working task descriptor is executed on both the primary storage processor and the secondary storage processor and without interruption by execution of any other task descriptor. Execution of the working task descriptor includes simultaneously transitioning through individual task states on both the primary storage processor and the secondary storage processor, and simultaneously executing functions on both the primary storage processor and the secondary storage processor during associated task states.

Task-based framework for synchronization of event handling between nodes in an active/active data storage system

Techniques are provided within a mapped RAID (Redundant Array of Independent Disks) system for assigning RAID extents to partnership groups and changing drive extent indications within RAID extents when splitting a group of storage drives into partnership groups. RAID extents are assigned to a RAID extent group corresponding to a newly created partnership group from a subset of RAID extents that contain RAID extents indicating higher total numbers of drive extents located in the physical data storage devices contained in the partnership group. When changing drive extent indications in RAID extents, new drive extents may be allocated to replace drive extents located outside of a partnership group such that a resulting variance in the values of a neighborhood matrix for the partnership group is minimized, to ensure that RAID extents are evenly distributed across the drives in the corresponding partnership group, to prevent drive rebuild performance from being compromised.

Assigning RAID extents and changing drive extent allocations within RAID extents when splitting a group of storage drives into partnership groups in a data storage system

The present invention relates to a method and system for providing parity protection in an object storage system. The present invention allows for tracking the storage requirements for chunks in a distributed storage cluster when transitioning from replica-based protection to parity or erasure coding-based protection and when transitioning from parity or erasure coding-based protection to replica-based protection.

Growing community of inventors

Northborough, MA, United States of America

Ilya Usvyatsky