International Business Machines Corporation

Globalfoundries Inc.

Sergey Marenkov

Jonathan Amit

Danny Harnik

Chaim Koifman

Dmitry Sotnikov

Ety Khaitzin

Nir Halowani

Rostislav Raikhman

Lilia Demidov

Amir Lidor

Yosef Shatsky

Ronen Itshak Kat

Asaf Porat-Stoler

Jonathan Fischer-Toubol

George Goldberg

Ben Sasson

Afief Halumi

Ori Shalev

Reut Cohen

Shai Ishaya Tahar

Yifat Kuttner

Tom Sivan

Yakov Gerlovin

Rivka Mayraz Matosevich

Constantine Gavrilov

Orit Nissan-Messing

Oded Margalit

Ofer Leneman

Aviv Caro

Christopher M Dennett

Lior Shlomov

Assaf Yaari

Kat I Ronen

Dolev Ish Am

Ury Matarazzo

Gil Shapira

Nir Friedman

Shai Taharlev

A mechanism is provided for dispersed location-based data storage. A request is received to write a data file to a referrer memory region in a set of memory regions. For each data chunk of the data file, responsive to a comparison of a hash value for the data chunk to other hash values for other stored data chunks referenced in the referrer memory region indicating that the data chunk fails to exist in the referrer memory region, responsive to the data chunk existing in another memory region in the set of memory regions, responsive to the memory region failing to be one of the predetermined number N of owner memory regions associated with the referrer memory region, and responsive to the predetermined number N of owner memory regions failing to have been met, a reference to the data chunk is stored in the referrer memory region.

Performance of dispersed location-based deduplication

A method and system may reconstruct data in a smart storage array where upon detection of data inconsistency, an application of the system is notified and affected data strips are rebuilt. When an initiator detects stripe corruption, the initiator may report the strip corruption to storage. The storage may lock the strip for I/O operations. Initiators may determine recovery scenarios for rebuilding the data strips and send the scenarios to storage. At storage, initiator replies may be collected, and a rebuild option with the highest votes may be automatically determined for a reconstruction operation. Once the reconstruction operation is completed, the rebuilt stripe may be unlocked and data operations may recommence.

Reconstructing data in a smart storage array

Embodiments for reducing rebuild time in a computing storage environment in by a processor. One or more disk drive failures in a Redundant Array of Independent Disks (RAID)-may be rebuilt by holding at least three parity strips per stripe while using one or more of the at least three parity strips according the one or more disk drive failures.

Reducing rebuild time in a computing storage environment

Embodiments for optimizing dual-layered data compression in a storage environment. In a data storage system having a primary compressor implemented in a storage controller and a secondary compressor implemented within a drive-enclosure, the primary compressor is selectively used to perform a first one of a plurality of actions on input/output (I/O) data while a second one of the plurality of actions is performed on the I/O data by the secondary compressor, thereby reducing latency and improving an overall compression performance while processing the I/O data.

Optimizing dual-layered compression in storage systems

A computer-implemented method according to one embodiment includes determining resource usage of at least a first module in a grid storage system having multiple modules and approximately equal resource usage across the multiple modules of the grid storage system. The computer-implemented method further includes determining a garbage collection cost in the grid storage system by stopping garbage collection in a second of the modules of the grid storage system, determining a resource usage in the second module upon stopping the garbage collection, and comparing the resource usage in the second module to the resource usage of the at least the first module. The method further includes adjusting an amount of garbage collection based on both the garbage collection cost and the resource usage.

Adaptive garbage collection (GC) utilization for grid storage systems

A computer-implemented method includes dividing a data set into a plurality of regions and dividing the plurality of regions into a plurality of chunks of fixed size. The computer-implemented method further includes determining a sample size of the plurality of chunks to be sampled for each region, wherein the sample size is determined based, at least in part, on an acceptance of a likelihood of identifying at least one collision between two regions corresponding to logical entities of a first cluster of logical entities. The computer-implemented method further includes sampling the plurality of chunks for each region based on the determined sample size. The computer-implemented method further includes generating a hash value for each chunk sampled and storing each hash value in an index. The computer-implemented method further includes identifying one or more collisions between the plurality of regions. A corresponding computer system and computer program product are also disclosed.

Identification of high deduplication data

A computer-implemented method includes receiving a set of basis fingerprints corresponding to image chunks within a basis set of image regions wherein each image region within the basis set of image regions comprises one or more image chunks, and generating a fingerprint for each image chunk of a plurality of selected image chunks within an unprocessed region of a machine image to produce a plurality of sampled fingerprints. The method also includes determining a similarity metric for the unprocessed region from the sampled fingerprints and the basis fingerprints, comparing the similarity metric for the unprocessed region with a selected threshold, and including the unprocessed region within the basis set of image regions in response to determining that the similarity metric is less than the selected threshold. A corresponding computer program product and computer system are also disclosed herein.

Deduplication ratio estimation using an expandable basis set

Embodiments for efficient garbage collection in a data storage environment. In a storage system comprising multiple storage devices having respective sets of storage regions, at least one respective storage fragmentation threshold used to trigger a garbage collection operation is identified. The garbage collection operation is performed to reclaim data space in the storage system according to each of a block perspective and an area perspective. The block perspective performs the garbage collection operation on individual blocks of data and the area perspective performs the garbage collection operation on a plurality of the blocks in a respective storage region. The block perspective and the area perspective portions of the garbage collection operation are executed independently of one another.

Efficient garbage collection in the presence of block dependencies

Embodiments for optimizing dual-layered data compression in a storage environment. In a data storage system having a primary compressor and a secondary compressor, the primary compressor is selectively used to perform a first one of a plurality of actions on Input/Output (I/O) data while a second one of the plurality of actions is performed on the I/O data by the secondary compressor, thereby reducing latency and improving an overall compression performance while processing the I/O data.

For efficient serialization of journal data using a processor device, serialization is combined with journaling of data into one data storage system on a write operation with subsequent consolidation for serialization of the journal data.

Efficient serialization of journal data

Methods, computing systems and computer program products implement embodiments of the present invention that include configuring a storage system to store multiple storage entities for access by one or more host computers in communication with the storage system, and specifying a compression condition including a minimum compression ratio. The storage system can then estimate an expected compression ratio for a given storage entity, compress the given storage entity upon the expected compression ratio meeting the compression condition, and provide, to a given host computer, access to the compressed given storage entity.

Storage data reduction analysis and forecast

A computer-implemented method includes receiving a set of representative machine image regions for a computing environment wherein the set of representative machine image regions collectively comprise a set of representative image chunks. The method also includes generating a fingerprint for each representative image chunk within the set of representative image chunks to produce a set of representative fingerprints, generating a fingerprint for selected image chunks within a measured machine image region to produce a set of sampled fingerprints, and determining a deduplication metric for the measured machine image region based on the representative fingerprints and the sampled fingerprints. A corresponding computer program product and computer system are also disclosed herein.

Deduplication ratio estimation using representative images

Various embodiments for data management in a replicated storage environment, by a processor device, are provided. In one embodiment, a method comprises storing a plurality of data replicas under a plurality of heterogeneous compression algorithms, wherein one of the data replicas is optimized for a data operation.

Heterogeneous compression in replicated storage

Growing community of inventors

Yehud, Israel

Sergey Marenkov