International Business Machines Corporation

Aviv Caro

Tzafrir Z Taub

Shay Haim Akirav

Ofer Leneman

Gil E Paz

Itay Maoz

Elena Drobchenko

Dov N Hepner

Uri Shmueli

Yariv Bachar

Asaf Levy

Chaim Koifman

Yosef Shatsky

Evgeny Budilovsky

Aharon Lazar

Asaf Ekshtein

Lior Aronovich

Jonathan Amit

David Darden Chambliss

Joseph S Glider

Oded Sonin

Dan Melamed

Liran Loya

Samuel Krikler

Shira Ben-Dor

Danny Harnik

Sergey Marenkov

Ben Sasson

Dmitry Sotnikov

Ety Khaitzin

Shai Ishaya Tahar

In one embodiment, a computer-implemented method for performing deduplication in conjunction with random read and write operations across a namespace that is divided into a plurality of disjoint regions is disclosed. The method includes: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing a deduplicated write operation. The deduplicated write operation includes: determining whether the disjoint regions comprise a second data chunk identical in content to a first data chunk; and in response to determining one or more of the plurality of disjoint regions other than the first region includes the second data chunk, establishing a reference from the first data chunk to the second data chunk. Corresponding systems and computer program products are also disclosed.

Region-integrated data deduplication

In one embodiment, a computer program product is configured for performing deduplication in conjunction with random read and write operations across a namespace divided into a plurality of disjoint regions. The computer program product includes a computer readable storage medium having program instructions embodied therewith, where the computer readable storage medium is not a transitory signal per se. The program instructions are executable by a computer to cause the computer to perform a method including: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager(s) of the plurality of disjoint regions, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing, by the computer, a deduplicated write operation of a first data chunk in a first region of the plurality of disjoint regions.

Exemplary embodiments for optimizing disaster recovery systems during takeover operations are provided. In one embodiment, by way of example only, a flag is set in a replication grid manager to identify replication grid members to consult in a reconciliation process for resolving intersecting and non-intersecting data amongst the disaster recovery systems for a takeover operation, including indicating those of the replication grid members that acquired ownership over cartridges belonging to source systems.

Optimizing disaster recovery systems during takeover operations

Calculating fingerprints for each one of a multiplicity of alignment combinations of fixed-size deduplication data blocks and comparing each of the fingerprints to stored deduplicated data fingerprints in a lookup database for determining a preferred deduplication data block alignment. A deduplication data block comprises each of the fixed-size deduplication data blocks.

Lookup-based data block alignment for data deduplication

A unique file-system node identification (ID) is created for each newly created node in a file system repository by combining a grid identification (ID), a repository identification (ID), and a node identification (ID) to form the unique file-system node ID. The unique file-system node ID is associated with a unique association identification (ID) thereby linking the node from a source repository to a target repository to form an association when performing a replication operation between the source repository and the target repository. Upon removing the association, the unique association ID is unlinked from a replication policy in a source repository, where an initiated replication operation is disabled.

Replica identification and collision avoidance in file system replication

A calculated factoring ratio is determined as a weighted ratio of current nominal data to physical data. A maximal nominal estimated space in the computing storage environment is calculated. A remaining space, defined as the maximal nominal estimated space minus a current nominal space in the computing storage environment, is calculated. Data replication operations are accepted and stored in the computing storage environment if the remaining space is below a predetermined threshold of space for backup operations.

Space reservation in a deduplication system

Exemplary method, system, and computer program product embodiments for optimizing disaster recovery systems during takeover operations are provided. In one embodiment, by way of example only, a flag is set in a replication grid manager to identify replication grid members to consult in a reconciliation process for resolving intersecting and non-intersecting data amongst the disaster recovery systems for a takeover operation. Additional system and computer program product embodiments are disclosed and provide related advantages.

A unique file-system node identification (ID) is created for each newly created node in a file system repository by combining a grid identification (ID), a repository identification (ID), and a node identification (ID) to form the unique file-system node ID. The unique file-system node ID is associated with a unique association identification (ID) for creating an association for linking the node from a source repository to a target repository when performing a replication operation between a repository and another repository. The different replication instances of the node in the file system repository are distinguished by the unique association ID assigned to each of the replication instances.

A calculated factoring ratio is determined as a weighted ratio of current nominal data to physical data based on at least one storage capacity threshold and a used storage space currently physically consumed by one of backup and replication data. A maximal nominal estimated space in the computing storage environment is calculated. A remaining space, defined as the maximal nominal estimated space minus a current nominal space in the computing storage environment, is calculated.

Pursuant to execution of a predetermined failback policy, if a storage device is not preexistent in a source storage system operable in the computing environment, and an owner of the storage device is one of a DR storage system and a storage system having previously withdrawn from a replication grid, and the DR storage system has authorization to transfer ownership of the storage device by a replacement operation, and an instance of the storage device is replicated to the source storage system to generate a replicated instance of the storage device on the source storage system.

Disaster recovery failback

Exemplary method, system, and computer program product embodiments for optimizing disaster recovery systems during takeover operations are provided. In one embodiment, by way of example only, a flag is set in a replication grid manager to identify replication grid members to consult in a reconciliation process for resolving intersecting and non-intersecting data amongst the disaster recovery systems for a takeover operation. The replication grid members are consulted for the takeover operation to accommodate a coordination of an ownership synchronization process for cartridges not distributed on-time to the replication grid members. Additional system and computer program product embodiments are disclosed and provide related advantages.

A deduplication storage capacity is estimated as a function of an expected deduplication ratio, the expected deduplication ratio being a combined average of a current deduplication ratio and a configured deduplication ratio, the current deduplication ratio depending on the data currently stored in the deduplication storage, and the configured deduplication ratio being an estimate made at a configuration stage of the deduplication computing storage environment.

Capacity management in deduplication storage systems

Exemplary method, system, and computer program product embodiments for changing ownership of cartridges, such as virtual cartridges between remotely located virtual tape libraries, are provided. In one embodiment, by way of example only, processes and protocols for the changing ownership of the cartridges are controlled from a primary location to a secondary location. The production site is moved for the cartridges. The ownership of the cartridges is waived. Access is allowed to the cartridges. Additional data is written and replicated using resources of the cartridges.

Changing ownership of cartridges

Exemplary system, and computer program product embodiments for changing ownership of cartridges, such as virtual cartridges between remotely located virtual tape libraries, are provided. In one embodiment, by way of example only, processes and protocols for the changing ownership of the cartridges are controlled from a primary location to a secondary location. The production site is moved for the cartridges. The ownership of the cartridges is waived. Access is allowed to the cartridges. Additional data is written and replicated using resources of the cartridges.

Methods, computer systems, and computer program products for deduplicating data are provided. Data is parsed to identify portions of metadata within the data. The data and identified portions of metadata are processed by a deduplication engine to be storable in a single repository. The deduplication engine is adapted for deduplicating the data without at least one of deduplicating and indexing the identified portions of metadata.

Growing community of inventors

Modiin, Israel

Aviv Caro