International Business Machines Corporation

Mihail Corneliu Constantinescu

Maohua Lu

Joseph S Glider

David Darden Chambliss

Abdullah Gharaibeh

David Allan Pease

Anurag Sharma

Prasenjit Sarkar

Aameek Singh

Eric Kevin Butler

Bhushan Pradip Jain

Reshu Jain

Subashini Balachandran

Dilip N Simha

Sandeep R Patil

Riyazahamad Moulasab Shiraguppi

Duane Mark Baldwin

Kavita Chavda

Jan Hendrik Pieper

Clodoaldo Barrera

Ronald Barthold Arps

Richard J Ayala, Jr

Adyasha Maharana

A method, computer system, and a computer program product for compressing at least one delta file for a plurality of genomic data files is provided. The present invention may include receiving the plurality of genomic data files as input. The present invention may also include determining a plurality of rows by traversing the received plurality of genomic data files. The present invention may then include comparing the plurality of rows associated with the traversed plurality of genomic data files. The present invention may further include generating a plurality of resulting delta files based on the compared plurality of rows. The present invention may also include compressing the generated plurality of resulting delta files by utilizing a general purpose file compressor.

Context aware delta algorithm for genomic files

Data deduplication for data storage tapes includes intercepting tape control commands for a single data storage tape. The intercepted tape control commands are modified for adding processing logic and parameters for placement of deduplicated file data on the single data storage tape. Deduplication metadata is written to a metadata portion of the single data storage tape. The deduplicated file data is written to a data portion of the single data storage tape based on the placement to increase read throughput for a deduplicated set of individual files and to reduce an average number of per-file gaps on the single data storage tape without re-duplicating deduplicated data for meeting optimization of individual file accesses.

Optimized data placement for individual file accesses on deduplication-enabled sequential storage systems

Data deduplication for data storage tapes includes intercepting tape control commands for a single data storage tape. The intercepted tape control commands are modified by adding processing logic and parameters for placement of deduplicated file data on the single data storage tape. Deduplication metadata is written to a metadata portion of the single data storage tape. The deduplicated file data is written to a data portion of the single data storage tape based on the placement to increase read throughput for a deduplicated set of individual files and to reduce an average number of per-file gaps on the single data storage tape without re-duplicating deduplicated data for meeting optimization of individual file accesses.

Detecting data duplication includes maintaining a fingerprint directory including one or more entries. Each entry includes a data fingerprint and a data location for a data chunk. A shadow list including a record of fingerprint values not contained in the fingerprint directory is maintained. Each entry is associated with a seen-count attribute, which is an indication of how often a data fingerprint has been seen in arriving data chunks to be written in a storage system, and distinguishes multiply-seen entries for data fingerprints present in at least two data chunks from once-seen entries for data fingerprints present in no more than a single data chunk. Each entry retrieved from the shadow list relates to twice-seen fingerprints.

Deduplicating storage with enhanced frequent-block detection

Data reduction in a storage system comprises determining attributes of data for storage in the storage system and determining expected data reduction effectiveness for the data based on said attributes. Said effectiveness indicates the benefit that data reduction is expected to provide for the data based on said attributes. The data reduction further comprises applying data reduction to the data based on the expected data reduction effectiveness and performance impact, to improve resource usage efficiency.

Enhancing performance-cost ratio of a primary storage adaptive data reduction system

Embodiments of the present invention provide an approach for migrating virtual machines across network (e.g., WAN) separated data centers (e.g., storage clouds). Specifically, under embodiments of the present invention, a first storage system associated with a first data center is synchronized with a second storage system associated with a second data center via a storage system link. Then, a minimal state of a virtual machine is migrated from a first computer in the first data center to a second computer in the second data center via a WAN link. Using the minimal state, the virtual machine is stored in the second computer. Thereafter, the storage system link is terminated. In addition, as updated pages are received in memory of the first computer, they are migrated to the second computer via the WAN link. Once this migration is complete, the WAN link can be terminated. Therefore, embodiments of the present invention provide at least two forms of synchronization: computational synchronization and storage synchronization.

Migrating virtual machines across network separated data centers

Migrating a sub-volume in data storage with at least two de-duplication domains, each of the domains having at least one sub-volume. A first sub-volume is assigned to a de-duplication domain and a first content summary is computed for the first sub-volume. Similarly, a second sub-volume is assigned to a second de-duplication domains and a second content summary is computed for the second sub-volume. A first content affinity is calculated between the first sub-volume and a third sub-volume, and a second content affinity is calculated between the second sub-volume and the third sub-volume. A domain placement is selected for the third sub-volume based on comparison of the first content affinity and the second content affinity.

De-duplication with partitioning advice and automation

Assignment of files to a de-duplication domain. Address space of data files is divided into multiple containers. For each of the containers, a file metadata scan is performed to obtain file system metadata, which is aggregated and summarized in a content feature summary. A content feature summary prediction measurement is measured between containers from the generated content feature summary, and files from each container are assigned to a de-duplication domain based upon the content similarity predication measurement.

De-duplication deployment planning

Data deduplication for data storage tapes comprises determining the read throughput of a deduplicated set of individual files on a single data storage tape, and determining a placement of deduplicated file data on a single data storage tape to reduce an average number of per-file gaps on the tape. Deduplicated file data is placed on the single data storage tape based on said placement to increase an average read throughput for a deduplicated set of individual files.

Embodiments of the invention relate to a method and computer program product for providing a scalable representation of metadata for deduplicated storage systems. The method includes identifying shared data segments that are contained in a plurality of data objects in a deduplicated storage system. A data object centric graph is generated. The generating includes creating vertices that represent the data objects and creating edges between the data objects. An edge connecting two data objects indicates that the two data objects contain at least one shared data segment in common. Each shared data segment between any two data objects is represented by at most one of the edges. At least one of the data objects is manipulated based on the data object centric graph.

Scalable graph modeling of metadata for deduplicated storage systems

Detecting data duplication comprises maintaining a fingerprint directory including one or more entries, each entry including a data fingerprint and a data location for a data chunk. Each entry is associated with a seen-count attribute which is an indication of how often the fingerprint has been seen in arriving data chunks. Higher-frequency entries in the directory are retained, while also taking into account recency of data accesses. A data duplication detector detects that the data fingerprint for a new chunk is the same as the data fingerprint contained in an entry in the fingerprint directory.

Methods, systems, and computer program products are provided for deduplicating data mapping a plurality of file blocks of selected data to a plurality of logical blocks, deduplicating the plurality of logical blocks to thereby associate each logical block with a corresponding physical block of a plurality of physical blocks located on a physical memory device, two or more of the corresponding physical blocks being non-contiguous with each other, determining whether one or more of the corresponding physical blocks are one or more frequently accessed physical blocks being accessed at a frequency above a threshold frequency and being referred to by a common set of applications, and relocating data stored at the one or more frequently accessed physical blocks to different ones of the plurality of physical blocks, the different ones of the plurality of physical blocks being physically contiguous.

High read block clustering at deduplication layer

Aspects of the invention are provided for decoding a selected span of data within a compressed code stream. A selection of data within the compressed code stream from an arbitrary position is presented for decompression. The arbitrary position is the starting point in the compressed code stream for decompression, and a phrase within the compressed code stream containing the starting point is identified. From the arbitrary starting point, a back pointer may provide direction to the literal. The literal is extracted as a decoding of the compressed data associated with the starting point.

Growing community of inventors

San Jose, CA, United States of America

Mihail Corneliu Constantinescu