Emc IP Holding Company LLC

Satish Visvanathan

Rahul Ugale

Mahesh Kamat

Yamini Allu

Balaji Subramanian

Debasish Garai

Ganesh Karuppur Rajagopalan

Dheer Moghe

Shuang Liang

Akshay Narayan Muramatti

Vrushali A Kulkarni

Ganesh Rajagopalan

A method, article of manufacture, and apparatus for creating a fingerprint to container id index is discussed. The index may be stored in-memory, on disk, and on a solid-state device. The index may be used to quickly locate a container identifier given a data segment fingerprint.

Solid state cache index for a deduplicate storage system

According to some embodiments, a backup storage system receives a request from a client to access a data segment. The system looks up an in-memory index for a first entry based on a fingerprint of the data segment, such that the in-memory index includes a number of entries, each entry mapping a fingerprint to a storage location of a solid state device (SSD) storing a corresponding data segment. In the event a first entry is found, the system retrieves the data segment from the SSD device based on the first entry. In the event of a read failure while accessing the SSD device, the system determines failed data blocks of the SSD device associated with the failure event and invalidates the failed data blocks of the SSD device such that subsequent access to the failed data blocks are denied without having to access a corresponding SSD device.

Method and system for accessing data stored in data cache with fault tolerance

According to some embodiment, a backup storage system receives a request from a client to read a data segment associated with a file object stored in a storage system. In response to the request, the system performs a lookup operation in a first index stored in a memory to identify a first index entry based on a fingerprint of the requested data segment to obtain a first write-evict unit (WEU) identifier (ID) identifying a first WEU storing the requested data segment. The system accesses a solid state device (SSD) operating as a cache memory device to retrieve the data segment from the first WEU. The system extracts and decompresses a compressed data segment retrieved from the first WEU and returns the decompressed data segment to the client without accessing a storage unit for retrieving the same data segment.

Accessing deduplicated data from write-evict units in solid-state memory cache

According to some embodiments, a backup storage system receives a request from a client at a storage system for accessing data segments. For each of a first groups of the data segments requested that are stored in a solid state device (SSD) cache, the system requests a first batch job for each of the first groups to retrieve the first groups of the data segments from the SSD cache via a first set of input/output (IO) threads. For each of a second groups of the data segments requested that are not stored in the SSD cache, the system requests a second batch job for each of the second groups to retrieve the second groups of the data segments from storage units of the storage system via a second set of input/output (IO) threads. The system assembles received segments and returns them to the client altogether.

Method for pipelined read optimization to improve performance of reading data from data cache and storage units

According to some embodiments, a backup storage system receives a request from a client for writing a data segment associated with a file object stored to a storage system. In response to the request, the system writes the data segment to one of many storage units of the storage system. The system determines whether the data segment is associated with a file region of the file object that is frequently accessed. The system writes the data segment in a first of many of write-evict units (WEUs) stored in a solid state device (SSD) operating as a cache memory device for caching data, after it is determined that the data segment is associated with the frequently accessed file region.

Method for efficient write path cache load to improve storage efficiency

A method, article of manufacture, and apparatus for locating data inside a deduplicated storage system is discussed. An index on a solid state device may contain a finger print to container identifier mapping. The container identifier may identify a logical container storing data related to the fingerprint. This data may therefore be located using the index given the fingerprint.

Method for locating data on a deduplicated storage system using a SSD cache index

According to some embodiment, a backup storage system receives a request from a client for writing a data segment associated with a file object stored to a storage system. In response to the request, the system writes the data segment to one of a plurality of storage units of the storage system. The system determines whether an amount of access to a solid state device (SSD) operating as a cache memory device exceeds an access threshold based on a time window distribution. The system throttles access to the SSD for caching data in response to determining that the amount of access to the SSD has exceeded the access threshold.

Method and system for window-based churn handling in data cache

According to some embodiment, a backup storage system determines whether one or more data segments associated with a file object have been updated, where the data segment(s) are stored in a first segment level in a multi-level hierarchy. For each updated data segment, the system applies a reference-based hierarchical data structure to track metadata associated with the updated data segment, where the metadata is stored in a second segment level in the multi-level hierarchy. The system determines whether the reference-based hierarchical data structure has reached a specific size. The system writes the metadata associated with the updated data segment(s) to a solid state device (SSD) operating as a memory cache device responsive to determining that the reference-based hierarchical data structure has reached the specific size.

Method and system for metadata churn absorption

According to some embodiment, a backup storage system receives a request from a client for reading a data segment associated with a file object stored in a storage system. In response to the request, the system determines whether a cache hit counter associated with the data segment exceeds a cache hit threshold. The system further determines whether the data segment is associated with a file region of the file object that is frequently accessed. The system writes the data segment into a memory responsive to determining that the cache hit counter exceeds the cache hit threshold and the data segment is associated with the frequently accessed file region. Otherwise, the system writes the data segment into a solid state device (SSD) operating as a cache device.

Method and system for memory-based data caching

A method, article of manufacture, and apparatus for maintaining a cache index inside a deduplicated storage system is discussed. A block allocation is requested from a persistent storage device, wherein the block allocation comprises a logical block for storing data. The logical block is determined to be unavailable. A next sequential block is requested from the persistent storage until an available logical block is located, the available logical block satisfying the block allocation. Data comprising a plurality of buckets is written to the available logical block.

Method for handling block errors on a deduplicated storage system

A method, article of manufacture, and apparatus for scaling an index on a solid state drive in a deduplicated storage system is discussed. The index may be scaled as index hash buckets approach capacity, or as new solid state drives are added to the system. The hash buckets may be mapped to HDD hash buckets, which are stored on a hard drive.

Scaling an SSD index on a deduplicated storage system

A method, article of manufacture, and apparatus for maintaining a cache index inside a deduplicated storage system is discussed. A determination is made to flush a cache buffer to the cache index, wherein both the cache buffer and the cache index comprise a plurality of cache buckets, wherein the cache buckets are stored on a plurality of logical data blocks. A write is initiated to at least one of the plurality of logical data blocks, wherein the write comprises at least one of the cache buckets. An error is received in response to the write request. The error is stored in a memory on the deduplicated storage system.

Method for maintaining a cache index on a deduplicated storage system

A method, article of manufacture, and apparatus for populating an index cache on a deduplicated storage system is discussed. A determination to flush an in-memory index to a hard drive ('HDD') on the deduplicated storage system is made, wherein the in-memory index comprises in-memory hash buckets containing fingerprint identifiers and container identifiers. A first HDD index is loaded from the HDD into a memory, wherein the first index includes a plurality of HDD buckets. The fingerprint identifiers and the container identifiers are merged from the in-memory hash buckets into the HDD buckets. The HDD buckets are mapped to a plurality of solid state drive ('SSD') buckets, the SSD buckets together comprising a SSD index. The fingerprint identifiers and container identifiers are inserted into the plurality of SSD buckets.

Method for populating a cache index on a deduplicated storage system

A method, article of manufacture, and apparatus for performing backup operations using a deduplicated storage system is presented. A data segment can be received at a deduplicated storage system. A fingerprint can be calculated for the data segment. The fingerprint and the offset can be stored in a reference cache on the deduplicated storage system. A metadata segment can be loaded from the deduplicated storage system. The metadata segment can be updated to include the fingerprint. Backup operations may include storage and management using a tree structure.

Method to improve the I/O performance in a deduplicated storage system

A method, article of manufacture, and apparatus for backup operations. A read request is received for a data segment at a deduplicated storage system. The data segment is determined not to be in a data cache on the deduplicated storage system. A data segment reference for the data segment is determined to be in a data segment reference cache on the deduplicated storage system. The data segment is read from a persistent storage using the data segment reference.

Growing community of inventors

San Jose, CA, United States of America

Satish Visvanathan