Intelliflash by Ddn, Inc.

Hgst, Inc.

Shailendra Tripathi

Sreekanth Garigala

Sandeep Sebe

Enyew Tan

Daniel P McGregor

Danny McGregor

Systems and methods are described for fully replacing an online first RAID group of data storage devices with a second RAID group of data storage devices, whereby the data is read in parallel from multiple devices in the first group and written in parallel to multiple devices in the second group. Sub-chunks of data chunks not currently storing any data may be bypassed from copying, as well as sub-chunks still storing data that is no longer in-use by a client. The data may be copied as-is rather than from and to the same logical block address on each respective group. A similar process may be applied for shrinking the number of storage devices allocating to a storage pool.

Fully replacing an existing RAID group of devices with a new RAID group of devices

A disclosed method for managing a RAID (redundant array of independent devices) file system memory comprises coalescing small read and write chunks into reasonably sized coalesced chunks up to 1 MB without any small size I/Os. The method also includes creating a new mapping layer configured to map the small size chunks in the coalesced chunks. The method additionally comprises flushing the coalesced chunks by allocating large chunks based on a chunk size coalescence or on a check point hit in terms of a SPA Sync. The method may also include applying a dynamic parity algorithm on the coalesced chunks to generate parities and data fragmented columns in reasonably big sized chunks. The method may yet include creating an appropriate coalesced chunk header to maintain the new mapping layer. The method may still comprise generating a chunked block pointer and a regular block pointer configured to co-exist in dual process threads.

Intelligent redundant array of independent disks

A process for reconstructing data stored on a failed storage drive in a RAID storage system involves performing a resilvering procedure on a first portion of the data for reconstruction and performing a recompaction procedure on the remaining portion of the data for reconstruction. Because the resilvering procedure rebuilds its data only on the replacement storage drive, but the recompaction procedure rebuilds its data on one or more, likely multiple, non-failed storage drives, the additional bandwidth provided by the non-failed drives is utilized to increase the speed of the overall data reconstruction. Determining how much of the data storage chunk processing to distribute to the resilvering and to the recompaction procedures may be based on the respective write workloads of the replacement and non-failed drives, as well as on the percentage of free storage space available from each of the chunks.

Intelligent redundant array of independent disks with resilvering beyond bandwidth of a single drive

A method, system and computer program product for managing an amount of storage written from a transaction group comprises creating a plurality of transaction classes within each of a plurality of write transaction classes and discovering an initial underlying drive capability assignable to the transaction classes when a storage pool is allocated (SPA). Each transaction class capability weight is adjusted via computing an instruction and operations per second, a data rate per second and a time slice at every storage pool allocator sync time. An unspent time slice from an adjusted transaction class is lent to a plurality of remaining transaction classes yet to be adjusted via a staged processing distribution during sync of the SPA. The disclosed method further includes assigning system resources and underlying drive capability to each transaction class per its capability weight based on a ratio of weights for each transaction class of operations.

Predictable client latency via background file system operations

A method, system and computer program product for managing a file system includes a plurality of allocation areas of loaded allocation trees to serve a request per file system drive for a transaction group. The system also includes a module configured to move a loaded allocation tree to an unloading tree in the event the loaded allocation tree will not satisfy the request. The system additionally includes a module configured to select and place in a loading tree queue a most eligible unloaded tree based on a weight of each unloaded tree. The system further includes a module to asynchronously process the loading tree queue and an unloading tree queue threads parallel to a storage pool allocator sync process. Allocation areas are attached in a circular ring of loaded allocation trees. Space map trees are converted into fixed size areas for constant latency loading logs and creating index trees.

Predictable allocation latency in fragmented log structured file systems

A system and process for recompacting digital storage space involves continuously maintaining a first log of free storage space available from multiple storage regions of a storage system such as a RAID system, and based on the first log, maintaining a second log file including a bitmap identifying the free storage space available from a given storage chunk corresponding to the storage regions. Based on the bitmaps, distributions corresponding to the storage regions are generated, where the distributions represent the percentage of free space available from each chunk, and a corresponding weight is associated with each storage region. The storage region weights may then be sorted and stored in RAM, for use in quickly identifying a particular storage region that includes the maximum amount of free space available, for recompaction.

Intelligent redundant array of independent disks with high performance recompaction

A method for managing file system memory includes starving an initial metadata memory area of a storage pool created for a file system transaction including cache data and log data. The method also includes determining on a transactional basis a ratio of metadata memory use to cache data use and an available unused portion of cache data memory replaceable in predetermined slab increments in the cache data. The method additionally includes shrinking transactionally the cache data per the slab increments based on the ratio via cleaning the cache data slabs and attaching the cache data slabs to the metadata memory area for temporary metadata use. The method further includes replacing the cache data slabs from the metadata memory area to the cache data based on a completion of the file system transaction. The slabs vary 5% to 10% or any arbitrary percentage of metadata space to cache data space.

Dynamic memory shrinker for metadata optimization

A method, system and a computer program product for managing file system memory includes a module configured to implement a separate replacement policy and a separate index for a persistent second level adaptive replacement cache (L2ARC) logically part of a first level ARC. The system also includes a module configured to cluster compressed chunks of data on multiple physical devices via aligning the clusters of data chunks on a byte boundary basis on each of the devices. The method additionally includes a module configured to create a storage pool allocator (SPA) to track the compressed and packed chunks on the multiple devices via an attached active page and attached multiple closed pages. The method further includes re-adding an evicted data from the L2ARC to an active page to be written again thereto based on a configurable threshold number of hits to data in the L2ARC via an L2ARC hit counter.

Growing community of inventors

Fremont, CA, United States of America

Shailendra Tripathi