Emc IP Holding Company LLC

Emc Corporation

Grant R Wallace

Philip N Shilane

Frederick Douglis

Mark L Huang

Cheng Li

Ming Benjamin Zhu

Stephen Smaldone

Nitin Garg

Hyong Shim

Kai Yi Li

Edward Kihyen Lee

Shuang Liang

Jianqiang Luo

Windsor Hsu

Fabiano C Botelho

Yamini Allu

Guanlin Lu

Radia Joy Perlman

Vladimir Mandic

Mark D Malamut

Scott Auchmoody

Mahesh Kamat

Mark Twomey

Xing Lin

Abhinav Duggal

Systems and methods for determining an access pattern in a computing system. Accesses to a file may contain random accesses and sequential accesses. The file may be divided into multiple regions and the accesses to each region are tracked. The access pattern for each region can then be determined independently of the access patterns of other regions of the file.

Input/output patterns and data pre-fetch

Systems and methods for managing content in a flash memory. A locking data structure is used to control access to data structures and the locking data structure is implemented in flash memory. The locking data structure is updated by overwriting the data such that the associated data structure is identified as locked or unlocked.

Locking structures in flash memory

Embodiments relating to garbage collection for a deduplicated and compressed storage device are described. One embodiment provides for a computer implemented method including creating a multiple sets of Bloom filters distributed across a set of multiple computing device nodes. One set of Bloom filters stores differing ranges of fingerprints for data stored on deduplicated storage containers, while a second set of Bloom filters includes a set of fingerprints correlated with files in a file system directory structure. A set of live fingerprints is determined for live fingerprints and storage segments associated with those fingerprints are copied to new storage containers.

Method and system for distributed garbage collection of deduplicated datasets

Systems and methods for managing content in a flash memory. Systems and methods for implementing hash tables in a flash memory are disclosed. A hash table may include a flat array or an array of buckets that are each associated with a linked list. Adding or removing entries from the hash table or from the linked list are achieved by performing an overwrite operation where possible to pointers affected by the table operation.

Hash tables in flash memory

Systems and methods for recovering corrupted data or missing data from a cache are provided. When a data corruption is discovered in a storage system, the cache may be searched to determine if a valid copy of the corrupted data can be recovered from the cache.

Cache based recovery of corrupted or missing data

Systems and methods for reducing input/output operations in a computing system that uses a cache. Input/output operations associated with cache index lookups are reduced by tracking the location of the requested data such that the data can be invalidated without having to access the cache index. Input/output operations can be reduced by invalidating the entry in the cache index when reading the corresponding data.

Reducing index operations in a cache

Systems and methods for managing content in a flash memory. A data structure such as a Bloom filter is implemented in flash memory such that updates to the data can be performed by overwriting pages in the memory.

Bloom filters in a flash memory

Techniques for balancing data compression and read performance of data chunks of a storage system are described herein. According to one embodiment, similar data chunks are identified based on sketches of a plurality of data chunks stored in the storage system. A first portion of the similar data chunks as a first group is associated with a first storage area. The first storage area is associated with one or more data chunks that are dissimilar to the first group but are likely accessed together. The first group of the similar data chunks and its associated dissimilar data chunks are compressed and stored in the first storage area.

System and method for balancing compression and read performance in a storage system

Systems and methods for a low-overhead index for a cache. The index is used to access content or segments in the cache by storing at least an identifier and a location. The index is accessed using the identifier. The identifier may be shortened or be a short identifier. Because a collision may occur, the index may also include one or more meta-data values associated with the data segment. Collisions can be resolved by also comparing the metadata of the segment with the metadata stored in the index. If both the short identifier and metadata match those of the segment, the segment is likely in the cache and can be accessed. Segments can also be inserted into the cache.

Low-overhead index for a flash cache

Systems and methods for managing data structures in a flash memory. A library is provided that supports read requests and write requests. The library allows reads and writes to be implemented without requiring the client to understand how the data structure is implemented in the flash memory.

Additive library for data structures in a flash memory

Systems and methods for managing content in a flash memory. Content or data in a flash memory is overwritten when the write operation only requires bits to be set. This improves performance of the flash and extends the life of the flash memory.

Flash interface for processing dataset

Systems and methods for rebuilding an index for a flash cache are provided. The index is rebuilt by reading headers of containers stored in the cache and inserting information from the headers into the index. The index is enabled while being rebuild such that lookup operations can be performed using the index even when the index is incomplete. New containers can be inserted into used or unused regions of the cache while the index is being rebuilt.

Systems and methods for rebuilding a cache index

Use of a survival queue to manage a container-based flash cache is disclosed. In various embodiments, a corresponding survival time is associated with each of a plurality of containers stored in a flash cache, each container comprising a plurality of data blocks. The survival time may be determined based at least in part on a calculated proportion of relatively recently accessed data blocks associated with the container is associated with the container. A container to evict from the flash cache is selected based at least in part on a determination that the corresponding survival time of the selected container has expired.

Container-based flash cache with a survival queue

Systems and methods for managing content in a flash memory. Data structures such as trees and graphs are implemented in a flash memory. Nodes or field nodes such as pointers or invalidation bits are updated using an overwrite operation where possible or by invalidating certain fields and temporarily storing changes in an in-memory table.

Trees and graphs in flash memory

Embodiments relating to garbage collection for a deduplicated and compressed storage device are described. One embodiment provides for a method comprising creating a first set of temporary files associated with a range of fingerprints for data within data files associated with a directory tree structure; creating a second set of temporary files associated with a range of fingerprints of storage segments stored on one or more deduplicated storage containers; sorting the fingerprints in each temporary file using distributed out of core sorting across each node in the set of multiple computing device nodes to generate a first set of sorted files and a second set of sorted files; determining an intersection of the fingerprints in the first set of sorted files and the second set of sorted files; and generating a garbage collection recipe for each of the one or more deduplicated storage containers.

Growing community of inventors

Pennington, NJ, United States of America

Grant R Wallace