Dell Products Inc.

Emc IP Holding Company LLC

Kunxiu Gao

Kuolin Hua

Malak Alshawabkeh

Michael J Scharland

Jiahui Wang

Evgeny Malkevich

Stephen Richard Ives

James Marriot Guyer

Peng Yin

Seema G Pai

Ningdong Li

Daohong Wang

Li Lang

In a drive cluster in which multiple parity EC (N+P) is implemented, rebuild-related reads are balanced across drives for recovery from drive failure. N members per protection group are read and (P−1) members are skipped, where skipping a member means omission from member rebuild calculations. Per-disk skip counts are calculated, and members that are eligible to be skipped are selected such that per-disk read counts are balanced.

Load balancing for erasure coding with multiple parities

A container system facilitates selective migrations of data from high-wear inducing data devices to spare data devices to promote SSD wear levelling. Storage capacity is configured into same-size cells with each container having the same number of cells as RAID width W. The cells are distributed over W SSDs. The containers are subdivided into equal-size data devices, each distributed over the same set of W SSDs as its associated container. Candidate migration source-target pairs are selected using heuristics guided by a prediction model that correlates disk wear rate with total writes of existing workloads. Wear rate predictions are computed using a weighted sum of write rates of all data devices on a disk. Wear rate predictions for candidate migration pairs are computed inclusive of additional wear caused by data migration to select a migration plan.

Solid-state disks wear leveling with predictive migration of data devices in a raid container system

A predictable distribution of protection group members and spares in compliance with RAID requirements is maintained as a drive cluster is scaled-up and eventually split into multiple clusters. Conditions under which addition of new disks using rotation-relocation and cell borrowing would create recurrent patterns of distribution of protection group members are recognized and the associated groups of disks are index-swapped to reduce the amount of data and parity information that needs to be relocated.

Leveraging recurrent patterns of RAID members distribution for efficient array growth

Sequentially ordered groups of same-size cells are distributed in a drive array representative matrix as group number=((row number−1)+(column number−1)) modulo (N), where N is the total number of drives. A plurality of the groups are reserved as spare groups. Responsive to a drive failure, one of the spare groups is selected, an ordered target list of spare cells is created, an ordered source list of cells containing protection group members on the failed drive is created, and the spare cells are assigned to the cells containing protection group members on the failed drive, where the protection group members are rebuilt. The target list starts at the spare cell having the minimum drive index that is greater than the failed drive index. The source list is ordered by either sequential cell indices or alphabetically by group-representing letters.

RAID configuration and spare capacity assignment with recoverability from multiple sequential failures

A minimal cluster of W+1 drives for RAID width W includes at least 2*W same-size sequentially indexed cells on each drive. More specifically, each drive is organized into W*(LCM of 2, 3, . . . N, N+1) same-size cells, where N is a predetermined number to support even distribution of spares over the first N cluster splits. Spare cells of aggregate capacity equivalent to a single drive are distributed such that there are W spare cells in each W-cell-wide submatrix. In the first cell index of each of the submatrices, a protection group is created in sequentially indexed drives starting with the lowest indexed drive. Also in each submatrix, W spare cells and (W−1) additional protection groups are distributed in cell-index-wise consecutive runs of a spare cell followed by sequentially ordered members of the additional protection groups. Spare capacity is widely distributed by adding drives to the cluster, creating spare cells, splitting the cluster into first and second clusters, and distributing the spare cells across W drives in each of the first and second clusters.

Growing and splitting raid clusters with wide distribution and assignment of spare capacity for fast parallel data recovery

New drives numbering fewer than a RAID width W are added to a primary drive array of K=W+1 sequentially-ordered original drives with same-size sequentially-ordered cells, where RAID groups are sequentially ordered and distributed such that a first cell of each sequentially-ordered drive contains a RAID group member in the sequential order of the protection groups and subsequent cells of each original drive in sequence contain members in the sequential order, wrapped. The new drives are added by selecting a subset of the original drives adjacent to the new drives such that the selected new drives and original drives total W, selecting a number of sequential cell indices equal to the number of new drives, and relocating protection group members in the selected sequential cell indices on the selected original drives to the new drives such that free space is created across W cells of each of the selected cell indices.

Fast and flexible data capacity upgrade via efficient reconfiguration

The disk drives of an array are organized into clusters of multiples of G+1 drives with G same-size indexed subdivisions, where G is the number of members in a protection group. G+1 groupings are created in each cluster, including G groupings distributed over G drives in single subdivision indices and one grouping distributed diagonally over multiple subdivision indices. A single grouping in at least one cluster is configured as spare capacity. Protection groups are located in the other groupings. Drive IO loading is rebalanced by swapping the location of a selected protection group with the location of the spare grouping. The protection group to be relocated may be selected by using LOF scores to identify a cluster with IO loading outlier drives that are overloaded and then calculating IO loading that would result from swapping locations of each protection group in that cluster with the spare grouping.

Disk array load balancing via interchangeable spare and data allocation

A drive subset matrix is created with at least N+1 drives each having N*N same-size subdivisions. Conceptually, N submatrices are created along with spares equivalent to at least one drive of storage capacity. The spares are located such that every drive has an equal number of spares +/−1. One protection group is located in a lowest indexed subdivision of each of the submatrices. Members of other protection groups are located by selecting members in round robin order and placing each selected member in a free subdivision having a lowest drive index and lowest subdivision index. The drive subset can be grown, split, and reorganized to restore balanced and efficient distribution of spares.

Successive raid distribution for single disk expansion with efficient and balanced spare capacity

In a storage system that implements RAID (D+P) protection groups a drive subset initially has (D+P) drives plus a spare drive with (D+P) splits. Spare splits are distributed with drive index and split index adjacency such that no single drive or split index contains multiple spare splits. When the drive subset is incremented by one drive a group of selected splits are relocated to the new drive based on drive index and split index adjacency such that no single drive or split index contains multiple members of a new protection group. If one of the drives is failing or fails, then an adjusted spare split index value is calculated for each protection group member on that drive so that the protection group members are rebuilt or relocated without placing more than one member of any protection group on a single drive. Adjusted spare split index values may be calculated in steps using the data split indices in ascending order and the largest drive indices in descending order.

Flexible raid sparing using disk splits

A storage system has a plurality of storage nodes having equal non-volatile storage capacity that is subdivided into equal size cells. Host application data that is stored in the cells is protected using RAID or EC protection groups each having members stored in ones of the cells and distributed across the storage nodes such that no more than one member of any single protection group is stored by any one of the storage nodes. Spare cells are maintained for rebuilding protection group members of a failed one of the storage nodes on remaining non-failed storage nodes so full data access is possible before replacement or repair of the failed storage node.

Resilient data storage system with efficient space management

One or more aspects of the present disclosure relate to recovering at least one failed disk. In embodiments, determining a storage reserve capacity allocated for recovering at least one storage device of a storage array is determined. Zero or more storage portions from each storage device of at least one storage cluster for disk recovery are adaptively assigned based on the storage reserve capacity. The failing and/or failed disk using the assigned storage portions is recovered in response to detecting a failing and/or failed disk.

Storage array disk recovery

The lifespans of the solid stated drives (SSDs) of a storage array are modelled using linear regression with monitored wear level and power-on time. The models predict when individual SSDs will reach a wear level corresponding to readiness for replacement. A drive replacement process makes efficient use of available empty drive slots to replace SSDs in batches. SSDs that are ready for replacement are ranked in terms of priority for replacement. If the number of SSDs that are ready for replacement exceeds the number of available empty drive slots, then ranking us used to assign individual SSDs to different batches for replacement.

Prediction and prioritization of solid-state drives replacement

Physical storage devices (PSDs) of a protection group cluster (PGC) may be represented by a protection group matrix (PGM) having a plurality of rows and a plurality of columns, where each row corresponds to a PSD of the PGC, and each column corresponds to a partition of each PSD. The value specified in each cell at an intersection of a row and column specifies the protection group of the PGC to which the partition of the PSD represented by the column and row, respectively, is (or will be) assigned. In response to one or more of PSDs being added to a PGC, the PGM may be reconfigured, including adding new rows, and transposing portions of columns to the new rows, or transposing portions of rows to portions of columns of the new rows. Protection members of the PGC may be re-assigned based on the reconfiguration.

Expanding raid systems

Managed drives of a storage node with different size drives in a fixed arithmetic relationship are organized into clusters of same size drives. Every drive is configured to have M*G same-size partitions, where M is a positive integer variable defined by the arithmetic relationship and G is the RAID group size. The storage capacity of all drives can be viewed as matrices of G+1 rows and M*G columns, and each matrix is composed of submatrices of G+1 rows and G columns. Diagonal spare partitions are allocated and distributed in the same pattern over groups of G columns of all matrices, for increasing partition index values. Members of RAID groups are vertically distributed such that the members of a given RAID group reside in a single partition index of a single cluster. When a drive fails, protection group members of the failed drive are rebuilt in order on spare partitions characterized by lowest partition indices for increasing drive numbers across multiple clusters. Consequently, drive access for rebuild is parallelized and latency is reduced.

Sharing spare capacity of disks with multiple sizes to parallelize RAID rebuild

A storage system has a plurality of heterogenous storage nodes characterized by non-uniform total non-volatile storage capacity. Storage capacity of all nodes is configured as same-size cells and represented as a set of matrices. The matrices have dimensions corresponding to consecutive cell indices and consecutive storage node indices. Initially, storage nodes having the same storage capacity are consecutively indexed so that the representative matrices are not ragged due to gaps, referred to herein as non-contiguous storage space, where cells do not exist because of differences in numbers of cells in adjacently indexed storage nodes. Addition of more heterogeneous storage nodes can create such gaps when the cells of those storage nodes are added to the matrices. The gaps are managed in the ragged matrix representations by coalescing the non-contiguous storage space, thereby enabling selected protection group members to be relocated using matrix mathematics such as linear algebra so that new protection groups can be created, and capacity can be scaled.

Growing community of inventors

Boxborough, MA, United States of America

Kunxiu Gao