Hewlett Packard Enterprise Development LP

Sonam Mandal

Sudhanshu Goswami

Umesh Maheshwari

Vinay Manivel

Hy Dinh Vu

Srivenkatesh Kumar Vaithianathan

In some examples, a system performs data deduplication using a deduplication fingerprint index in a hash data structure comprising a plurality of blocks, wherein the hash data structure is stored in persistent storage, and a block of the plurality of blocks comprises fingerprints computed based on content of respective data units. The system uses an indirect block in a memory to access a given block of the plurality of blocks in the hash data structure, the indirect block containing references to blocks of the hash data structure containing the deduplication fingerprint index, and the references indicating storage locations of the plurality of blocks in the persistent storage.

Indirect block containing references to blocks of a persistent fingerprint index

In some examples, a system performs data deduplication using a fingerprint index comprising a plurality of buckets, each bucket of the plurality of buckets comprising entries associating fingerprints for data units to storage location indicators of the data units, wherein a storage location indicator of the storage location indicators provides an indication of a storage location of a data unit in persistent storage. For adding a new fingerprint to the fingerprint index, the system detects that a corresponding bucket of the plurality of buckets is full, in response to the detecting, adds space to the corresponding bucket by taking a respective amount of space from a further bucket of the plurality of buckets, and inserts the new fingerprint into the corresponding bucket after increasing the size of the corresponding bucket.

Increasing sizes of buckets of a fingerprint index

In some examples, a system receives, in a synchronization buffer stored in a memory, fingerprint index entries for respective data units received by the system, each fingerprint index entry of the fingerprint index entries comprising a fingerprint computed for a corresponding data unit, and a storage location indicator for the corresponding data unit. The system merges the fingerprint index entries in the synchronization buffer with fingerprint index entries of a persistent fingerprint index in a hash data structure stored in persistent storage, the hash data structure comprising a plurality of buckets.

Merging buffered fingerprint index entries

In some examples, a system dynamically adjusts a proportion of incoming data blocks for which respective fingerprints are added to a fingerprint index, where the fingerprint index correlates fingerprints for data blocks to storage location indicators for the data blocks. The system inputs, into a buffer, fingerprint index delta updates for the incoming data blocks according to the dynamically adjusted proportion, and updates the fingerprint index based on the fingerprint index delta updates input into the buffer. The system retrieves, in response to a fingerprint of an incoming data block, a collection of block entries from a block location index that stores information for data blocks in temporal order, and deduplicates the incoming data blocks using the collection of block entries from the block location index.

Growing community of inventors

Santa Clara, CA, United States of America

Sonam Mandal