International Business Machines Corporation

Ori Shalev

Jonathan Amit

Ben Sasson

Sergey Marenkov

Nir Halowani

Lilia Demidov

Chaim Koifman

Yosef Shatsky

Jonathan Fischer-Toubol

Nir Milstein

Omer Haklay

Yuval Berger

Rostislav Raikhman

While discharging a data chunk the chunk is compressed into a storage block. If the chunk is found to be too large to be able to be completely compressed into the storage block certain characteristics of the data chunk are examined to determine whether or not the data chunk should be split. If the data chunk should be split, a remaining portion of the data chunk is compressed to a storage block that is next in chronological order to the original storage block. If the data chunk should not be split, all of the data chunk is moved to the next chronological storage block while leaving any remaining space in the storage block as unused.

Reducing fragmentation in compressed journal storage

While discharging a data chunk, the chunk is compressed into a storage block. If the chunk is found to be too large to be able to be completely compressed into the storage block, certain characteristics of the data chunk are examined to determine whether or not the data chunk should be split. If the data chunk should be split, a remaining portion of the data chunk is compressed to a storage block that is next in chronological order to the original storage block. If the data chunk should not be split, all of the data chunk is moved to the next chronological storage block while leaving any remaining space in the storage block as unused.

Delay commands are injected into sequential input/output (I/O) requests and the effects of the injected delay commands are measured for determining whether a storage system handling the sequential input/output (IO) requests is causing a bottleneck, while the processing of sequential I/O requests is delayed for a time period when injecting the delay command.

Self-detecting storage bottleneck while handling sequential I/O operations

For efficiently storing and retrieving data and metadata in phases, in a first phase, metadata tokens, which are assigned to metadata-emitting entities, are used for storing the data and the metadata together in a single input/output operation while piggybacking the metadata of least active metadata-emitting entities onto one of the metadata-emitting entities having one of the metadata tokens. In a second phase, the metadata is re-written to a metadata delta journal for reclaiming the metadata tokens. In a third phase, the metadata journal is applied to a metadata structure containing the metadata of the storage system, the metadata delta journal is then cleared after successfully updating the main metadata structure with the metadata of the metadata journal. The metadata journal is swapped with an empty metadata journal for concurrently adding metadata while retaining the metadata journal until applying the metadata delta journal to the metadata structure.

Efficiently storing and retrieving data and metadata

For combining virtual mapping metadata and physical space mapping metadata in a storage system by a processor device in a computing environment, data and metadata are maintained into separate virtual streams. The separate virtual streams include a metadata stream for the metadata and a data stream for the data. Metadata for each input/output (I/O) operation received is determined using a linear function operation, the function operation being an offset of the metadata in the metadata stream that is equal to the I/O operation multiplied by a maximal metadata ratio. The metadata is allocated on the metadata stream and the metadata stream is divided into fixed size block that is responsible for describing a size of a logical space, where the logical space is equal to one divided by the maximal metadata ratio, and it is determined if the metadata has been previously loaded.

Combining virtual mapping metadata and physical space mapping metadata

Systems capable of transformation of logical data objects for storage and methods of operating thereof are provided. One method includes identifying among a plurality of requests addressed to the storage device two or more 'write' requests addressed to the same logical data object, deriving data chunks corresponding to identified 'write' requests and transforming the derived data chunks, grouping the transformed data chunks in accordance with the order the requests have been received and in accordance with a predefined criteria, generating a grouped &#x201c;write&#x201d; request to the storage device, and providing mapping in a manner facilitating one-to-one relationship between the data in the obtained data chunks and the data to be read from the transformed logical object. The method further includes obtaining an acknowledging response from the storage device, multiplying the obtained acknowledging response, and sending respective acknowledgements to each source that initiated each respective &#x201c;write&#x201d; request.

Systems and methods for transformation of logical data objects for storage

Systems capable of transformation of logical data objects for storage and methods of operating thereof are provided. One method includes identifying among a plurality of requests addressed to the storage device two or more 'write' requests addressed to the same logical data object in a distributed network, deriving data chunks corresponding to identified 'write' requests and transforming the derived data chunks, grouping the transformed data chunks in accordance with a predefined criteria, generating a grouped &#x201c;write&#x201d; request to the storage device, and providing mapping in a manner facilitating one-to-one relationship between the data in the obtained data chunks and the data to be read from the transformed logical object. The method further includes obtaining an acknowledging response from the storage device, multiplying the obtained acknowledging response, and sending respective acknowledgements to each source that initiated each respective &#x201c;write&#x201d; request.

Method and system for transformation of logical data objects for storage

Various embodiments for storing a logical object are provided. In one such embodiment, by way of example only, incoming data is divided corresponding to a logical data object into a plurality of independent streams, associating each data chunk of a plurality of obtained data chunks with a corresponding stream among the plurality of independent streams. At least one of the obtained data chunks and derivatives thereof is sequentially accommodated in accordance with an order the obtained chunks are received, while keeping the association with the corresponding streams. A global index is generated as a single meta-data stream accommodated in the logical data object and comprising information common to the plurality of independent streams and related to mapping between data in the logical data object and the obtained data chunks.

Dividing incoming data into multiple data streams and transforming the data for storage in a logical data object

For combining virtual mapping metadata and physical space mapping metadata in a storage system by a processor device in a computing environment, data and metadata are maintained into separate virtual streams. The separate virtual streams include a metadata stream for the metadata and a data stream for the data. Information of a mapping of logical data to a location on the data stream and a current state of unused spaces in the data stream is linearly distributed within the metadata stream.

Recovering data in a logical object utilizing an inferred recovery list

Intercepting, transforming, and grouping data objects for storage

Intercepting, transforming, and grouping transformed data objects for storage

Transforming data objects and queuing the transformed data objects for storage

There is provided a system capable of transformation of logical data objects for storage and method of operating thereof. The method comprises: a) identifying among a plurality of requests addressed to the storage device two or more 'write' requests addressed to the same logical data object; b) deriving data chunks corresponding to identified 'write' requests and transforming the derived data chunks; c) grouping the transformed data chunks in accordance with the order the requests have been received and in accordance with a predefined criteria; d) generating a grouped &#x201c;write&#x201d; request to the storage device; and e) providing mapping in a manner facilitating one-to-one relationship between the data in the obtained data chunks and the data to be read from the transformed logical object. The method further comprises obtaining an acknowledging response from the storage device; multiplying the obtained acknowledging response, and sending respective acknowledgements to each source which initiated respective &#x201c;write&#x201d; request.

Transforming logical data objected for storage includes identifying multiple write request to the same logical object and grouping transformed data chunks

Delay commands are injected into sequential input/output (I/O) requests and the effects of the injected delay commands are measured for determining whether a storage system handling the sequential input/output (IO) requests is causing a bottleneck.

Growing community of inventors

Kiryat Ono, Israel

Ori Shalev