International Business Machines Corporation

Globalfoundries Inc.

Jonathan Fischer-Toubol

Sergey Marenkov

Asaf Porat-Stoler

Afief Halumi

Yosef Shatsky

Danny Harnik

Ety Khaitzin

Jonathan Amit

Ori Shalev

Shai Ishaya Tahar

Nir Milstein

Reut Cohen

Tom Sivan

Omer Haklay

David Darden Chambliss

Rivka Mayraz Matosevich

Ronen Gazit

Constantine Gavrilov

Chaim Koifman

Joseph S Glider

Orit Nissan-Messing

Dmitry Sotnikov

Nir Halowani

Christopher M Dennett

Michael Moser

Yifat Kuttner

Lior Shlomov

Ariel Waizel

Assaf Yaari

Gil Shapira

A mechanism is provided for dispersed location-based data storage. A request is received to write a data file to a referrer memory region in a set of memory regions. For each data chunk of the data file, responsive to a comparison of a hash value for the data chunk to other hash values for other stored data chunks referenced in the referrer memory region indicating that the data chunk fails to exist in the referrer memory region, responsive to the data chunk existing in another memory region in the set of memory regions, responsive to the memory region failing to be one of the predetermined number N of owner memory regions associated with the referrer memory region, and responsive to the predetermined number N of owner memory regions failing to have been met, a reference to the data chunk is stored in the referrer memory region.

Performance of dispersed location-based deduplication

A method and system may reconstruct data in a smart storage array where upon detection of data inconsistency, an application of the system is notified and affected data strips are rebuilt. When an initiator detects stripe corruption, the initiator may report the strip corruption to storage. The storage may lock the strip for I/O operations. Initiators may determine recovery scenarios for rebuilding the data strips and send the scenarios to storage. At storage, initiator replies may be collected, and a rebuild option with the highest votes may be automatically determined for a reconstruction operation. Once the reconstruction operation is completed, the rebuilt stripe may be unlocked and data operations may recommence.

Reconstructing data in a smart storage array

A computer-implemented method according to one embodiment includes receiving data, storing chunks of the data in fixed size blocks in physical storage, and deduplicating a copy of the data. The storing includes splitting some of the data chunks into portions for filling the fixed size blocks. Metadata is stored with the data chunks for chaining the associated portions of the split data chunks together. The deduplicating including creating a pointer to a first of the data chunks. A computer program product for utilizing free space in physical storage, according to another embodiment includes a computer readable storage medium having program instructions embodied therewith. The computer readable storage medium is not a transitory signal per se. The program instructions are readable and/or executable by a computer to cause the computer to perform the foregoing method.

Utilization of tail portions of a fixed size block in a deduplication environment by deduplication chunk virtualization

A computer-implemented method according to one embodiment includes determining resource usage of at least a first module in a grid storage system having multiple modules and approximately equal resource usage across the multiple modules of the grid storage system. The computer-implemented method further includes determining a garbage collection cost in the grid storage system by stopping garbage collection in a second of the modules of the grid storage system, determining a resource usage in the second module upon stopping the garbage collection, and comparing the resource usage in the second module to the resource usage of the at least the first module. The method further includes adjusting an amount of garbage collection based on both the garbage collection cost and the resource usage.

Adaptive garbage collection (GC) utilization for grid storage systems

A computer program product, according to one embodiment, includes a computer readable storage medium having program instructions embodied therewith. The computer readable storage medium is not a transitory signal per se. The program instructions are readable and/or executable by a processor to cause the processor to perform a method which includes: receiving a request to delete a volume stored in one or more regions in physical space of a storage system; determining whether at least one of the regions having at least a portion of the volume includes reclaimable space; deleting the portion of the volume from the at least one region having the reclaimable space in response to determining that at least one of the regions having at least a portion of the volume includes reclaimable space; and failing the received request to delete the volume in response to determining that none of the regions include reclaimable space.

Space reclamation in data deduplication storage systems

Methods, computing systems and computer program products implement embodiments of the present invention that include configuring, at a first time, a clustered storage system comprising multiple modules to store and process storage requests for respective sets of data objects. While processing the storage requests, respective subsets of the data objects having one or more data errors are identified, and at a second time subsequent to the first time, respective modules storing each of the data objects having at least one data error are identified. computing, based on the identified modules, a frequency distribution of the identified data errors in the data objects over the multiple modules. Based on the frequency distribution, a failure in a given module is identified, and the identified module is removed from the storage system. In some embodiments, prior to the second time, the data objects can be redistributed among the modules of the storage system.

Isolating faulty components in a clustered storage system with random redistribution of errors in data

Various embodiments for identifying data in a data deduplication system, by a processor device, are provided. In one embodiment, a method comprises efficiently identifying duplicate data in the data deduplication system by identifying fingerprint matches using a direct inter-region fingerprint lookup to search for the fingerprint matches in at least one of a plurality of metadata regions, the direct inter-region fingerprint lookup supplementing a central fingerprint index.

Direct lookup for identifying duplicate data in a data deduplication system

Various embodiments for data compression by a processor. Levels of data distribution are configured for data processing, including a first level of the data distribution incorporating a GRID network of data storage nodes, and a second level of the data distribution incorporating a GRID network of compressive nodes in communication with the GRID network of data storage nodes. Input/output (I/O) for an associated storage volume is load balanced between the data storage nodes, as data passes through the first level into the second level to be compressed or uncompressed.

Three dimensional grid compression storage

Delay commands are injected into sequential input/output (I/O) requests and the effects of the injected delay commands are measured for determining whether a storage system handling the sequential input/output (IO) requests is causing a bottleneck, while the processing of sequential I/O requests is delayed for a time period when injecting the delay command.

Self-detecting storage bottleneck while handling sequential I/O operations

Exemplary method, system, and computer program product embodiments for efficient one-pass cache-aware compression are provided. In one embodiment, by way of example only, an output of a fast compressor to Huffman encoding for achieving the one-pass cache-aware compression by using a predetermined Huffman-tree upon determining by the fast compressor a final representation of each data byte.

Efficient one-pass cache-aware compression

Delay commands are injected into sequential input/output (I/O) requests and the effects of the injected delay commands are measured for determining whether a storage system handling the sequential input/output (IO) requests is causing a bottleneck.

Growing community of inventors

Tel Aviv, Israel

Jonathan Fischer-Toubol