Emc IP Holding Company LLC

Emc Corporation

Christopher Alan Seibel

Philippe Armangau

Jean-Pierre Bono

Yubing Wang

Sitaram D Pawar

Bruce E Caram

Yining Si

John F Gillono

Sorin Faibish

Vamsi K Vankamamidi

Dennis Thomas Duprey

William C Davenport

Ivan Bassov

Srinivasa Rao Vempati

Soumyadeep Sen

Alexei Karaban

Maher Kachmar

Diane Delgado

James McCoy

Christopher Jones

Vladimir Shveidel

Ahsan Rashid

Socheavy D Heng

Jun Guo

Ivan Basov

Yingchao Zhou

Michael P Wahl

Monica Chaudhary

Rustem Rafikov

Henry Austin Spang, Iv

Jia Zhai

Vasily Olegovich Zalunin

Wengang Wang

Peter E Tolvanen

Gregory S Schaffer

Lawrence Yetto

A method, computer program product, and computing system for defining a token pool size for a storage system token pool associated with a storage system. An input/output (IO) processing load for the storage system may be determined. The token pool size for the storage system token pool associated with the storage system may be dynamically adjusted based upon, at least in part, one or more changes in the IO processing load determined for the storage system.

System and method for dynamically adjusting priority-based allocation of storage system resources

A method, computer program product, and computer system for virtualizing, by a computing device, a physical metadata space into a virtual metadata space. A translation table from the virtual metadata space to the physical metadata space may be implemented. Metadata in the physical metadata space may be cached based upon the virtual metadata space. The metadata in the physical metadata space may be moved without updating references associated with the metadata in the physical metadata space.

System and method for moving metadata without updating references used by the metadata

A technique for performing writes in a storage system that supports data compression tests incoming writes to determine whether the amount of data to be written is less than a predetermined threshold. If so, the storage system avoids compression and stores the amount of data as received without compressing it. If not, the storage system performs compression and stores the incoming data as a compressed extent.

Inline compression with small-write compression avoidance

A method is used in managing inline data de-duplication in storage systems. The method receives a request to write data at a logical address of a file in a file system of a storage system. The method determines whether the data can be de-duplicated to matching data residing on the storage system in a compressed format. Based on the determination, the method uses a block mapping pointer associated with the matching data to de-duplicate the data. The block mapping pointer includes a block mapping of a set of compressed data extents and information regarding location of the matching data within the set of compressed data extents.

Managing inline data de-duplication in storage systems

Embodiments are directed to techniques for performing deduplication. A method includes (a) obtaining a digest of a data block logically-positioned within a filesystem, the digest providing a hash value of data of the data block, (b) searching a Most Wanted Digest Cache (MWDC) within system memory for the digest, (c) locating an entry in the MWDC using the digest, wherein this locating indicates that the data block has the same data as another data block located elsewhere within the filesystem, the other data block having been previously persistently-stored, the entry having been added to the MWDC in response to the other data block having been deduplicated at least a plurality number of times, (d) locating a mapping structure referenced by the entry located from the MWDC, the mapping structure providing metadata about the other data block, and (e) deduplicating the data block and the other data block with reference to the located mapping structure.

Speeding deduplication using a most wanted digest cache

A storage processor in a data storage system includes a compression selection component that selects a data compression component to be used to compress host I/O data that is flushed from a persistent cache of the storage processor based on a current fullness level of the persistent cache. The compression selection component selects compression components implementing compression algorithms having relatively lower compression ratios for relatively higher current fullness levels of the persistent cache, and selects compression components implementing compression algorithms having relatively higher compression ratios for relatively lower current fullness levels of the persistent cache.

Compression of host I/O data in a storage processor of a data storage system with selection of data compression components based on a current fullness level of a persistent cache

A data storage system in which a transaction is generated that indicates at least one data block of a logical volume to be written to non-volatile data storage of a data, and in which the logical volume is accessible to multiple nodes in the data storage system. A system-wide lock is obtained for each data block indicated by the transaction. A new generation identifier is then created that is equal to a last transaction identifier that was created and stored during processing of a previously completed transaction. Each data block indicated by the transaction is stored into the non-volatile data storage of the data storage system together with the new generation identifier and the last transaction identifier is updated before each system-wide lock on each data block indicated by the transaction is released.

Generating and storing monotonically-increasing generation identifiers

A technique for managing metadata in a data storage system includes receiving a set of data to be stored in a file system of the data storage system and calculating values of metadata blocks that the file system will use to organize the set of data in the file system. The technique aggregates values of the metadata blocks and sends them in a single, atomic transaction to a persistent cache. The transaction either succeeds, in which case the persistent cache stores the values of all of the metadata blocks, or it fails, in which case the persistent cache stores none of the values of the metadata blocks. Over time, the persistent cache flushes the values of the metadata blocks to a set of non-volatile storage devices that back the metadata blocks in the data storage system.

Managing file system metadata using persistent cache

Methods and apparatus are provided for reporting fragment filling in storage systems. An exemplary method comprises obtaining at least one compressed allocation unit of data in a storage system; reserving space for the at least one compressed allocation unit in a compressed segment based on a greater of (i) a size of the at least one compressed allocation unit, and (ii) a minimum target fragment length specified for at least one write operation class; and writing the at least one compressed allocation unit to the reserved space. An attempt is optionally first made to fill the at least one compressed allocation unit of data in a free extent and/or a hidden fragment of at least one compressed segment prior to reserving the space. The reserving the space is optionally only performed if the attempt to fill the at least one compressed allocation unit of data is not successful.

Fragment filling for storage system with in-line compression

A technique for processing IO (Input/Output) requests includes executing a first set of instructions for processing an IO request using a first thread on a CPU core and providing the first thread with an option, based on how busy the CPU core is, either (i) to execute a second set of instructions for further processing the IO request itself or (ii) to pass the IO request to a second thread on the CPU core, such that the second thread executes the second set of instructions instead of the first thread.

Adaptive thread processing of IO requests

A technique for storing data in a data storage system detects that a read is being performed pursuant to a data copy request. In response, the data storage system stores a digest of the data being read in an entry of a digest cache. Later, when a write pursuant to the same copy request arrives, the storage system obtains the entry from the digest cache and completes the write request without creating a duplicate copy of the data.

Optimizing inline deduplication during copies

A method of protecting a filesystem against metadata corruption while maintaining high performance is provided. The method includes (a) receiving a storage command directing a computing device to store a block of file data to the filesystem and (b) in response: (1) writing the block to a data portion spanning a set of drives using a single-fault tolerant level of RAID, wherein writing the block includes performing exactly two write operations to the set of drives; and (2) writing filesystem metadata associated with the block to a metadata portion spanning a subset of the set of drives using a double-fault tolerant data redundancy scheme different than the single-fault tolerant level of RAID, the subset including at least three drives, the data and metadata portions being non-overlapping, wherein writing the metadata includes performing at least three write operations to the subset of the set of drives.

Techniques for optimizing metadata resiliency and performance

A technique for performing deduplication on compressed data in a file system operates inline with storage requests, e.g., prior to writing compressed data specified in those storage requests to persistent file system structures. As requests to write data arrive, the file system compares those data with data already stored in the file system. If a match is found, metadata structures are written but compression and file system storage of the newly arrived data is avoided.

Inline deduplication of compressed data

A technique for storing data selectively tags write requests received by a data storage system based on whether each write request is directed to a file having a file type on an exclusion list, thereby producing a set of tagged write requests directed to a file type on the exclusion list and a set of untagged write requests directed to a file type not on the exclusion list. The method further includes persistently storing tagged data specified by the set of tagged write requests without first compressing the tagged data, and persistently storing untagged data specified by the set of untagged write requests after compressing the untagged data.

Write tagging for selective avoidance of inline compression

Techniques for processing unaligned IO requests in data storage systems that provide optimization of inline compression. The disclosed techniques employ an unaligned IO cache, which is used by a data storage system to process unaligned IO requests containing data with sizes that are not multiples of a predetermined block size. By employing the unaligned IO cache while processing such unaligned IO requests, the data storage system can reduce the number of read-modify-write sequences required to process a sequential load of unaligned IO requests, thereby reducing the burden on processing resources of the data storage system.

Growing community of inventors

Walpole, MA, United States of America

Christopher Alan Seibel