Hewlett Packard Enterprise Development LP

Emc Corporation

Emc IP Holding Company LLC

Srikant Varadan

Dheer Moghe

Fabiano C Botelho

Sazzala Reddy

Umesh Maheshwari

Ying Xie

Nitin Garg

Tomasz Barszczak

Hao Luo

Gurunatha Karaje

Guilherme Vale Ferreira Menezes

Rajat Sharma

Nitin Muppalaneni

Senthil Kumar Ramamoorthy

Hy Dinh Vu

Steve Rodrigues

In some examples, a system maintains mapping metadata for data in storage, the data divided into chunks, the mapping metadata comprising chunk entries mapping chunk identifiers of the chunks to storage locations, wherein the chunk identifiers are monotonically increasing as chunks are created. The system updates an epoch in response to triggering a creation of a snapshot of the data. In response to a deletion of a first chunk, the system updates a first chunk entry for the first chunk to include a creation epoch identifier and a deletion epoch identifier for the first chunk. In response to a deletion of a first snapshot of the multiple snapshots, the system uses the creation epoch identifier and the deletion epoch identifier to determine whether to remove the first chunk entry from the mapping metadata.

Chunk and snapshot deletions

Examples described herein relate to selection and variation of extent size thresholds in a storage volume. Examples may select an extent size threshold for a volume according to a type of application that is to store data on the volume. Examples may dynamically vary the extent size threshold based on data reduction metrics, such as deduplication ratio and/or compression ratio. Examples may dynamically vary the extent size threshold for the volume based on cache performance metrics, such as cache miss rate. Examples may dynamically vary the extent size threshold for the volume based on an amount of dead storage space corresponding to partially overwritten extents in the volume.

Variable extent size

A technique for reducing recovery time in a computing apparatus includes: storing data in a key-value store residing in a persistent memory, the key-value store including a plurality of persistent nodes including key-value pairs, the key-value pairs of the persistent nodes being unsorted; caching selected persistent nodes in a volatile memory, the key-value pairs of the cached nodes being sorted, the caching including instantiating a dual pointer for each cached node, the dual pointer including a volatile pointer to a child node of the cached node and a persistent pointer to the child node of the corresponding persistent node in the key-value store; and rebuilding the cache of selected nodes upon demand using the dual pointers in the event of recovery.

Key-value store on persistent memory

Exemplary methods, apparatuses, and systems maintain a plurality of summary data structures corresponding to a plurality of logical file system namespaces representing a plurality of hierarchies of one or more directories having one or more files, each file being stored in the storage system as a plurality of segments in a deduplicated manner. In response to a request to estimate a storage usage by a first of the file system namespace, identify a first of the summary data structures corresponding to the first file system namespace, wherein the first summary data structure stores information summarizing deduplicated segments referenced by one or more files of the first file system namespace. Estimate the storage usage of the first file system namespace based on the first summary data structure and a global summary data structure, wherein the global summary data structure stores information summarizing deduplicated segments referenced by all of the file system namespaces.

System and method for determining physical storage space of a deduplicated storage system

When data blocks of a data seglet are compressed using a shared dictionary and when the requested data block (or blocks) do not include the last data block of the data seglet, an optimization in the read path may involve decompressing a certain portion of the data seglet from a starting position of the data seglet to a decompression endpoint of the data seglet, but not including the portion of the data seglet following the decompression endpoint. Such technique may involve the storing of a mapping that maps, for each data block within the data seglet, an identifier of the data block to a decompression endpoint that indicates a portion of the data seglet that includes the data block.

Methods and systems for efficiently reading a data block from a data seglet with compressed data blocks

Methods, systems, and computer programs are presented for deduplicating data. One method includes an operation for receiving a data block having a logical address. The storage system includes a logical address mapping index for mapping logical addresses to block numbers, a block index for mapping block numbers to physical locations, and a fingerprint index for mapping fingerprints of data blocks to block numbers. Additionally, the method includes an operation for inline processing the data block. Inline processing the data block includes determining a fingerprint of the data block, examining the fingerprint index to determine if the fingerprint is already mapped to an existing data block in the storage system, if the fingerprint is already mapped then adding a mapping of the logical address to the existing data block in the logical address mapping index, and if the fingerprint is not already mapped then creating the corresponding entries in the indices.

Processing of incoming blocks in deduplicating storage system

A garbage collector of a storage system traverses a namespace of a file system of the storage system to identify segments that are alive in a breadth-first manner. The namespace includes information identifying files that are represented by segments arranged in a plurality of levels in a hierarchy, where an upper level segment includes one or more references to one or more lower level segments, and at least one segment is referenced by multiple files. All live segments of an upper level are identified before any of live segments of a lower level are identified. Upon all live segments of all levels have been identified, the live segments are copied from their original storage locations to a new storage location, and a storage space associated with the original storage locations is reclaimed.

Method and system for garbage collection of data storage systems

Scaling Asynchronous reclamation of free space in de-duplicate multi-controller storage systems is disclosed. In one example, a set of de-duplicated data segments/blocks organized in a hierarchical data structure is received, the set of de-duplicated data segments/blocks are allocated across a plurality of controllers of the file system, where each de-duplicated data segment is restricted to be co-allocated to the same controller as the reference(s) referencing the de-duplicated data segment/block in the hierarchical data structure. In various embodiments, the file system is traversed, each controller of the file system updates its own local live vector and reclaim storage space that are not marked as alive.

Growing community of inventors

San Jose, CA, United States of America

Srikant Varadan