International Business Machines Corporation

Karrthik Kalaga Gopalakrishnan

Venkateswara Rao Puvvada

Saket Kumar

Ashish Pandey

Muthu Annamalai Muthiah

Ashish Chaurasia

Ravi Kumar Komanduri

Archana Chinnaiah

Saipriya M R

Jijo Varghese

A method for minimizing data transfer and storage utilization on cloud object storage systems is disclosed. In one embodiment, such a method replicates a file from a production system to a cloud object storage system. The method determines whether a number of hard links associated with the file is greater than one. In the event the number is greater than one, the method creates, on the cloud object storage system, a special object for the file and associates the file with the special object. Upon creating a hard link on the production system in association with the file, the method replicates the hard link from the production system to the cloud object storage system without replicating data associated with the file. A metadata reference to the special object is added to the hard link on the cloud object storage system. A corresponding system and computer program product are also disclosed.

Minimizing data transfer and storage utilization on cloud object storage systems

Disclosed are techniques for multi-protocol multi-site file replication, including a method comprising capturing, at a gateway device, IO operations performed on a filesystem, the filesystem having inode bits. The method may further comprise enqueuing the captured IO operations onto a plurality of queues, wherein each of the plurality of queues is associated with one of a plurality of target sites; and independently managing replication of the IO operations to the target sites using the inode bits.

Multi-protocol multi-site replication

A computer-implemented method according to one embodiment includes receiving, on a first cluster site, a first I/O request to migrate a plurality of filesets from a second cluster site to the first cluster site. The first cluster site includes a plurality of gateway nodes. The method further includes identifying at least two of the gateway nodes having resources available to perform operations of the migration, and hashing information of a plurality of filesets against the identified gateway nodes. The information includes inode numbers of entities that are mounted during fulfillment of the first I/O request. Operations of the first I/O request are distributed to the identified gateway nodes based on the hashing, and the identified gateway nodes are instructed to fulfill the operations.

Hashing information of an input/output (I/O) request against a plurality of gateway nodes

A computer-implemented method includes migrating an item to a cache cluster from a home cluster in a storage system in a cutover process and initiating a data validation process with the home cluster. The method includes enabling at least one tunable parameter on the item on the cache cluster in the storage system. The tunable parameter is configured to adjust the data validation process with the home cluster. The tunable parameter includes at least one condition. The method includes enabling at least one flag on the item on the cache cluster in the storage system. The flag is configured to indicate that the at least one condition of the tunable parameter on the item in the storage system is met. In response to the at least one condition of the tunable parameter being met, the method includes ending the data validation process with the home cluster in the storage system.

Migration of uncached data after cutover

Provided are techniques for role reversal of primary and secondary sites with minimal replication delay. During a backup phase, an Input/Output (I/O) operation selected from a group comprising a create operation and a rename operation is received with one or more local attributes of a local file. The I/O operation is processed to generate one or more remote attributes. The one or more local attributes are stored with the one or more remote attributes. During a reversed phase, a write operation for the remote file is processed, and, the write operation is sent to the primary replication system with the one or more local attributes, where the primary replication system uses a local file handle in the one or more local attributes to access the local file for processing the write operation to sync the local file with the remote file.

Role reversal of primary and secondary sites with minimal replication delay

A computer-implemented method according to one embodiment includes receiving, on a first cluster site, extended attributes associated with a first data operation where the first data operation was previously performed on data of a filesystem of second cluster site. In response to a second data operation being performed on data of a filesystem of the first cluster site, extended attributes associated with the second data operation are stored to a predetermined file of the filesystem of the first cluster site. The method further includes outputting, to the second cluster site, the extended attributes stored to the predetermined file of the filesystem of the first cluster site, and in response to a determination that a predefined event has occurred on the second cluster site, using the received extended attributes to fulfill a third data operation.

Exchanging extended attributes between different cluster sites in a clustered filesystem environment

A computer-implemented method according to one embodiment includes initializing a new gateway node at a first site in response to a failure of an old gateway node at the first site; creating, at the new gateway node, a list of dirty directories at the first site; synchronizing, with a second site by a background thread at the first site, all directories within the list of dirty directories; and synchronizing, with the second site by a foreground thread at the first site, new operations performed within the first site after the initialization of the new gateway node.

Recovering from a failure of an asynchronous replication node

A computer-implemented method according to one embodiment includes, in response to a determination that a predetermined operation has been performed on an object of a first file stored on a first cluster site, storing predetermined information about the object of the first file stored on the first cluster site. The predetermined information is stored on an extended attribute of the first file stored on the first cluster site. In response to a determination that the predetermined operation is performed on an object of a first file stored on a second cluster site, the predetermined information is removed from the extended attribute of the first file stored on the first cluster site. In response to a determination that a failure event has occurred on a queue of the first cluster site, a predetermined recovery process is performed, thereby enabling fulfillment of entries of the queue of the first cluster site.

Recovery from a clustered file system queue failure event using a modified extended attribute of a file

A computer-implemented method, according to one approach, includes: receiving a data operation request which includes an activated compound operation flag. The data operation request is added to a queue in a gateway node, and the data operation request is eventually transmitted to a disaster recovery site. An inode entry which corresponds to the portion of data is locked, and metadata associated with the inode entry is updated to indicate that the data operation request has been performed at the disaster recovery site. Supplemental data operation requests which correspond to the portion of data are also identified by evaluating the metadata associated with the inode entry. These supplemental data operation requests are transmitted to the disaster recovery site, and the metadata associated with the inode entry is updated to indicate that the supplemental data operation requests have been performed at the disaster recovery site. Furthermore, the inode entry is unlocked.

Using inode entries to mirror data operations across data storage sites

A computer-implemented method, according to one embodiment, includes: receiving, by a computer, a data access request; sending, by the computer, a recall request to a remote storage location for data which corresponds to the data access request; and receiving, by the computer, a copy of an existing object which includes blocks. The data which corresponds to the data access request is stored in at least one of the blocks. The data access request is satisfied, by the computer, by providing the copy of the existing object. Moreover, a sparse object, which only includes ones of the blocks which contain data that has been modified, is received by the computer. The sparse object is sent, by the computer, to the remote storage location; and one or more instructions to use the blocks included in the sparse object to update the existing object are also sent by the computer.

Growing community of inventors

Kodaikanal, India

Karrthik Kalaga Gopalakrishnan