International Business Machines Corporation

Michael G Sisco

Yu Meng Li

Howard Newton Martin

Matthew Joseph Anglin

Arthur John Colvig

Ya Jie Li

Colin Scott Dawson

Yin Xiang Xiong

Jason D Hagar

Jeremy M Bares

Robert G Genis, Jr

Diem T Nguyen

Xin Xu

Yan Xin Li

In one embodiment, a method includes receiving a query related to a first data extent. The method also includes determining whether the first data extent is eligible for deletion based on a first sliding window time period equal to a predetermined retention period added to a predetermined deletion cushion in response to the query requesting a deletion operation for the first data extent. The method also includes determining whether the first data extent is eligible for linkage based on a second sliding window time period equal to the retention period less a predetermined linkage cushion in response to the query requesting a linkage operation for the first data extent. The linkage cushion includes less time than the deletion cushion.

Lockless management of deduplicated data using reference tags

Provided are a computer program product, system, and method for considering input/output workload and space usage at a plurality of logical devices to select one of the logical devices to use to store an object. A determination is made of a logical device to store the object based on workload scores for each of the logical devices indicating a level of read and write access of objects in the logical device and space usage of the logical devices. The object is written to the determined logical device.

Considering input/output workload and space usage at a plurality of logical devices to select one of the logical devices to use to store an object

In one embodiment, a method includes creating a data extent catalog table including information related to data extents stored to computer readable storage media, receiving a query related to a first data extent, and determining whether the data extent is stored to the one or more computer readable storage media based on analysis of the data extent catalog table. Moreover, the method includes determining whether the first data extent is eligible for deletion based on a first sliding window time period. In addition, the method includes determining whether the first data extent is eligible for linkage based on a second sliding window time period.

A server computational device maintains commonly occurring duplicate chunks of deduplicated data that have already been stored in a server side repository via one or more client computational devices. The server computational device provides a client computational device with selected elements of the commonly occurring duplicate chunks of deduplicated data, in response to receiving a request by the server computational device from the client computational device to prepopulate, refresh or update a client side deduplication cache maintained in the client computational device.

Synchronization of a server side deduplication cache with a client side deduplication cache

The present invention relates to a method, system, and computer program product for data backup, the method comprising: performing first chunking on current data by using the same chunking method as that used by original backup data to obtain a current chunk; calculating hash value of the current chunk; and acquiring, from a hash value table of the original backup data, an identifier of a matched chunk whose hash value is the same as the calculated hash value of the current chunk, and incrementing number of continuous matched chunks by one. Since the pertinence between original backup data and current data is maximally utilized, performance of de-duplication method can be efficiently improved.

Method and system for data backup

Provided are a computer program product, system, and method for deduplicating chunk digests received for chunks in objects in objects provided by clients to store in a storage. An index has chunk signatures calculated from chunks of data in the data objects in the storage. A transaction requests a shared lock for a chunk digest calculated from a chunk in an object to add to the storage. In response to not receiving the shared lock, the transaction is granted a shared lock to the chunk digest in wait mode. The transaction does not proceed until being granted the shared lock. In response to receiving the shared lock, a determination is made as to whether the chunk digest is in the index. A reference to the chunk data is provided for the object when the chunk digest is indicated in the index.

Deduplicating chunk digests received for chunks in objects provided by clients to store

In a data protection field, a method for storing data in a data deduplication system, comprising: obtaining data chunks achieved from data deduplication; assigning the data chunks to at least one group; recording grouping information of the data chunks; with respect to each group, calculating parity data chunks based on data chunks in the group, where the parity data chunks are used for, in response to a data chunk in the group being damaged, causing the damaged data chunk to be recovered on the basis of other data chunks in the group and parity data chunks of the group; and storing the calculated parity data chunks. Also provided is an apparatus for storing data and a data deduplication system. The technical solution provided herein facilitates occupying as little physical storage space as possible while reducing the risk of the spread of data loss caused by the data deduplication technology.

Data duplication that mitigates storage requirements

A method and device for data de-duplication, comprising: performing data chunk partition on a current data object by using a different standard in each of a plurality of logical passes; searching one or more first redundant data chunks of the current data object in each logic pass based on the data chunks partitioned on the current data object in the logical pass, respectively, and performing data de-duplication on the current data object based on all of the found first redundant data chunks of the current data object. Other embodiments of the present invention may also relate to a data de-duplication system and a corresponding computer program product.

Data de-duplication

A server computational device maintains commonly occurring duplicate chunks of deduplicated data that have already been stored m a server side repository via one or more client computational devices. The server computational device provides a client computational device with selected elements of the commonly occurring duplicate chunks of deduplicated data, in response to receiving a request by the server computational device from the client computational device to prepopulate, refresh or update a client side deduplication cache maintained in the client computational device.

A reference counter corresponding to a base chunk of a plurality of chunks of a deduplicated data object is maintained, where the reference counter is incremented in response to an insertion of any chunk that references the base chunk, and where the reference counter is decremented, in response to a deletion of any chunk that references the base chunk. A queue is defined for processing dereferenced chunks of the plurality of chunks. The dereferenced chunks in the queue are processed in a predefined order, to free storage space.

Prioritization mechanism for deletion of chunks of deduplicated data objects

In each of a number of passes to deduplicate a data object, a transaction is started. Where an offset into the object has previously been set, the offset is retrieved; otherwise, the offset is set to reference a beginning of the object. A portion of the object beginning at the offset is deduplicated until an end-of-transaction criterion has been satisfied. The transaction is ended to commit deduplication; where the object has not yet been completely deduplicated, the offset is moved just past where deduplication has already occurred. The object is locked during each pass; other processes cannot access the object during each pass, but can access the object between passes. Each pass is relatively short, so the length of time in which the object is inaccessible is relatively short. By comparison, deduplicating an object within a single pass prevents other processes from accessing the object for a longer time.

Growing community of inventors

Vail, AZ, United States of America

Michael G Sisco