International Business Machines Corporation

Vast Data Ltd.

Kaminario Technologies Ltd.

Eli Malul

Shachar Fienblit

Yogev Vaknin

Lior Klipper

Olympia Gluck

Renen Hallak

Itzhack Goldberg

Michael Edward Factor

Amir Sasson

Gail Andrea Spear

Kenneth Wayne Boyd

Robert Francis Bartfai

Eyal Gordon

Sam Clark Werner

Gal Ashour

Amiram Hayardeny

Ifat Nuriel

Dalit Tzafrir

Noa Privman-Horesh

Aviad Zlotnick

Carol Santich Mellgren

Asaf Levy

Nicolas Marc Clayton

Thomas Charles Jarvis

Gilad Hitron

Eran Mann

Alon Berger

Ilan Steinberg

Niko Farhi

Shahar Salzman

A method for compressing an evaluated chunk (EC), the method includes (i) selecting a compressing scheme (SC) of a set of candidate SCs to provide a selected SC; and (ii) compressing the EC using the selected SC to provide a compressed EC. A similar chunk that is similar to the EC is tied to a reference chunk (RC) by having the similar chunk compressed based on the RC. The selecting includes evaluating candidate SCs of the set. The selecting is responsive to a first relationship between (i) a similarity score indicative of a similarity between the EV chunk and the similar chunk, and (ii) a similarity score indicative of a similarity between the similar chunk and the RC. A first candidate SC of the set includes untying the similar chunk from the RC, and tying the EV to the similar chunk. A second candidate SC of the set includes self-compressing the EV without tying the EV to the similar chunk.

Similarity based compression

A method for striping, the method may include performing, for each period of time out of multiple periods of time, the steps of: (i) determining striping rules; wherein the determining of the striping rules may include selecting one or more selected stripe size values out of multiple stripe size value candidates; wherein the selecting is based on values of storage system parameters that are obtained when applying the multiple stripe size value candidates; wherein the storage system parameters comprise storage space utilization and storage system throughput; and (ii) applying the striping rules by the storage system, during the period of time; wherein the applying comprises obtaining data chunks; converting the data chunks to stripes having at least one of the one or more selected stripe size values; and storing the stripes in the storage system.

Striping in a storage system

A method for generating a reconstructed version of a filesystem entity, the method may include (i) generating fallback retrieval metadata for a reconstructed version segment, when the reconstructed version segment is (a) associated with a corresponding intermediate version segment, and (b) the corresponding intermediate version segment is preceded by a corresponding most updated segment that does not exceed a fallback version of the filesystem entity; wherein the reconstructed version segment, the corresponding intermediate version segment and the corresponding most updated segment that does not exceed a fallback version have a same address range; (ii) generating a non-existing indicator for the reconstructed version segment, when the reconstructed version segment is (a) associated with the corresponding intermediate version segment, and (b) the corresponding intermediate version segment is not preceded by any corresponding most updated segment that does not exceed the fallback version; and (iii) maintaining intermediate metadata for retrieving one or more intermediate versions of the filesystem entity.

Segregating cold stripe defragmentation from new writes based on creation time of the cold stripe

A method for striping based on evaluated rules, the method may include determining a compatibility, with a storage system utilization policy, of storing stripes under evaluated rules; wherein the evaluated rules define a stripe size, a number of parity chunks per stripe, and maximal numbers of chunks within a stripe per different failure domains of different size ranges; checking whether the storing of the stripes is compatible with the storage system utilization policy; when finding that the storing of the stripes is not compatible then searching for one or more changes of one or more of the maximal numbers that yields compliant one or more maximal numbers that once applied results in a compliance with the storage system utilization policy; applying the compliant one or more maximal numbers when finding the compliant one or more maximal numbers; and determining that the evaluated failure domain rules are non-compliant when failing to find the compliant one or more maximal numbers.

Striping based on failure domains rules

A system and method for erasure coding. The method includes distributing a plurality of data chunks according to a mirroring scheme, wherein the plurality of data chunks is distributed as a plurality of rows among a plurality of non-volatile memory (NVM) nodes, wherein the mirroring scheme defines a plurality of groups, each group including a subset of the plurality of data chunks, wherein each data chunk in a group has a role corresponding to a relative position of the data chunk within the group, wherein data chunks included in the plurality of groups having the same relative positions within their respective groups have the same role, wherein each row of the plurality of rows includes at least one summation data chunk that is a function of at least one data chunk included in the row and of at least one extra data chunk included in at least one other row.

Techniques for improved erasure coding in distributed storage systems

A distributed storage system and a method for providing resiliency in distributed storage systems. The distributed storage system includes a plurality of storage nodes including a plurality of disks, wherein the plurality of disks includes a plurality of blocks, wherein the plurality of disks is logically segmented into the plurality of stripes, wherein each of the plurality of stripes is dynamically allocated to a portion of the plurality of blocks distributed across a subset of the plurality of disks; and a plurality of compute nodes, wherein each of the plurality of compute nodes is configured to read data from each of the plurality of storage nodes, wherein each of the plurality of compute nodes is assigned at least one stripe of the plurality of stripes, wherein each of the plurality of compute nodes is configured to write data to each stripe assigned to the compute node.

Resiliency schemes for distributed storage systems

The presently disclosed subject matter includes various inventive aspects, which are directed to direct read access of a host computer device to a share storage space in a data storage system, as well as control of the direct read of the host computer device by a control computer device in the data storage system.

Direct read control in a data storage system

A method for re-establishing a backup relationship between first and second volumes associated with one or more storage media in a data storage system is provided. The method comprises storing first information for preserving the backup relationship, in response to or in advance of receiving a request to terminate the backup relationship; recording changes to the first or second volumes that occur subsequent to terminating the backup relationship; and re-establishing the backup relationship between the first and second volumes according to the first information such that the first and second volumes are synchronized by merging the recorded changes with the first or second volumes.

Managing backup relationships in a data storage system

A method for consistent version of multiple information items is provided. The method includes receiving a conditional request to copy a version of multiple information items at a condition fulfillment point of time that is associated with a fulfillment of a condition of the conditional request. Then determining that the condition is fulfilled, and participating in generating a condition fulfillment point in time version of the multiple information items.

Conditional storage of multiple information items

Provided are a method, system, and article of manufacture for managing writes received to data units that are being transferred to a secondary storage as part of a mirror relationship. Synchronization information indicates data units to transfer from a primary storage to a secondary storage, wherein the synchronization information data is updated to indicate the transfer of one of the data units from the primary storage to the secondary storage. A storage controller receives a write request having an update to a target data unit comprising one of the data units indicated in the synchronization information that has not been transferred from the primary storage to the secondary storage. The write request is allowed to apply the update to the target data unit in response to transferring the data in the target data unit to a network adapter of the storage controller and before the data in the target data unit is transferred to the secondary storage.

Determining when to apply writes received to data units being transferred to a secondary storage

Generating a consistent point in time copy of data in a source volume and a target volume is achieved responsively to a first data modification request by writing a first altered version of the data onto a single source volume, asynchronously transferring the first altered version from the first storage site to a target volume located at a remote second storage site, while avoiding copying the first altered version onto other volumes at the first storage site. While asynchronously transferring the first altered version de-queuing a second modification request, and responsively to the second modification request synchronously transferring the first altered version from the first storage site to the target volume. Then a second altered version of the data is written to the single source volume and a copy transferred to the target volume in like manner.

System, method and computer program product for generating a consistent point in time copy of data

A method for generating a consistent point in time copy of data, the method includes: selecting at least one selected data block to be copied from a source volume to a target volume in response to a request to generate a consistent point in time copy of multiple data blocks; waiting until the source volume is ready to send the at least one selected data block to a remote volume while queuing at least one data block modify request; de-queuing the at least one queued modify requests while copying the at least one selected data block from the source volume to the target volume; wherein the copying includes utilizing a first copying mechanism to copy a first selected data block if a request to modify the first selected block is de-queued before the first selected data block is copied to the target volume; else, the copying includes using a second copying mechanism that is slower than the first copying mechanism.

An apparatus, system, and method are disclosed for activating a synchronous mirror as a primary storage volume. The apparatus system and method include directing a third storage volume to store updates sent to the third storage volume from a second storage volume onto a first storage volume in response to the first storage volume becoming operational after a failure, terminating sending updates from the second storage volume to the third storage volume in response to the first storage volume becoming substantially consistent with the third storage volume, and synchronously storing updates received by the second storage volume on the first storage volume and asynchronously storing updates received by the first storage volume on the third storage volume in response to terminating sending updates from the second storage volume to the third storage volume after the first and third storage volumes are substantially consistent.

Growing community of inventors

Tirat Carmel, Israel

Eli Malul