Emc IP Holding Company LLC

Oracle America, Inc.

Dell Products Inc.

Emc Corporation

William R Stronge

David Meiri

Xiangping Chen

Whay Sing Lee

Benjamin W Yoder

Rodney A Dekoning

Charles D Binford

Felix Shvaiger

Bhaskar Bora

Archana Parasnis

Richard W Meyer

Mark J Halstead

Deepak Vokaliga

Jeffrey Wilson

Anton Kucherov

Michael Ferrari

Sandeep Chandrashekhara

Howard Hall

Milan Merhar

Ian Adams

Christopher Cochrane

Stephen Ferrari

Christopher Sayles

Snapshots from a first LSU (R1) on a first storage system (A1) may be replicated to a second replica LSU (R2) on a second storage system (A2), for example, concurrently to remotely replicating (e.g., synchronously) write operations for R1 to R2. A process, P, on A1 executing the replication of the snapshots from R1 to R2 may be a separate process than the one or more processes on A1 executing remote replication of write operations for R1 to R2. During a consistency window on A1, outstanding write operations for R1 at the time the consistency window opened may be logged, and a pair of snapshots, SS1and SS1may be activated on R1 and R2, respectively. After the consistency window has closed, the SS1snapshot metadata and snapshot data may be updated based on the outstanding write operations.

Synchronous remote replication of snapshots

Snapshots may be used to implement remote replication, for example, asynchronously, between a first storage system, A1, and a second storage system, A2. A1 may take a first snapshot, SS2, of a logical storage unit, R1. After the snapshot SS2it taken, any dependent write operations associated with SS2may be reconciled, and differences between SS2and the last snapshot for R1, SS1, may be determined and recorded as a difference set. One or more replication instructions for R1 that include the write operations (or data and metadata corresponding thereto) of the difference set may be sent from A1 to A2. A2 may apply the differences to R2, and then take (activate) a snapshot of R2, SS2, which is a replica of SS2. After A2 activates SS2, A2 may send an acknowledgement to A1 indicating that SS2has been activated, and A2 may take a next snapshot of R1.

Snapshot-based remote replication

Data may be replicated from a host storage system to a target storage system. It may be determined to replicate a first logical storage element on the source storage system to a second logical storage element on the target storage system, wherein the first logical storage element defines a first data portion having a first value. It may be determined that a third logical storage element on the target storage system defines a second data portion having the first value. The first logical storage element may be replicated to the second logical storage element by establishing a deduplication relationship between the second logical storage element and the third logical storage element on the target storage system without transmitting the first data portion from the source storage system to the target storage system.

Remotely replicating duplicated data

A target-less point in time image (snapshot) of a storage volume is allowed to be built after activation, by enabling the snapshot data to be modified to create a crash-consistent replica of the source data after the snapshot has been activated. The data of the snapshot remains immutable from a user standpoint, but the snapshot is able to be quickly activated before all of the data of the snapshot has been included in the snapshot, to thus reduce an amount of time IO operations on the source volume are quiesced. A first snapshot of a storage volume is created on a primary storage system and a corresponding second snapshot of the storage volume is activated on a backup storage system before all the data of the first snapshot is received at the backup storage system. Entries of the activated second snapshot are then changed to point to correct back-end allocations.

Modifying data of activated target-less point-in-time images of storage volumes

An aspect includes determining a host write rate based on a number of chunks written to a storage volume during a first replication cycle and determining a target rate based on an estimated number of chunks written during a second replication cycle. The target rate is determined by dividing the number of chunks written during the second replication cycle by a recovery point objective (RPO) period. An aspect also includes transmitting chunks written during the second replication cycle at the target rate.

Replication link smoothing using historical data

First information for a first event of a computer system is captured, including first event parameters. A first event key is generated, based on the first event parameters. The first information and first key are both stored as a first event in a storage structure, in which the first event key indexes the first event and the captured first information. When second information is captured for a second event, a second event key is generated based on second event parameters of the captured second event. If the second event key matches the first event key, then the first event stored in the event collection system is dynamically updated to store the second event information as part of the first event, to minimize additional storage space needed in the storage structure for the second event.

Method for collecting performance data in a computer system

A system, computer program product, and computer-executable method of providing a layout-independent cryptographic stamp of a distributed data set from a data storage system, the system, computer program product, and computer-executable method comprising receiving a request for a cryptographic stamp of the distributed data set, creating a hash of each slice of the distributed data set, and using each hash from each slice of the distribute data set to create the cryptographic stamp of the distributed data set.

Layout-independent cryptographic stamp of a distributed dataset

A first storage system in one illustrative embodiment is configured to participate in a replication process with a second storage system. Each of a plurality of storage nodes of the first storage system comprises a plurality of storage devices and a set of processing modules configured to communicate with corresponding sets of processing modules on other ones of the storage nodes. In conjunction with the replication process, a first one of the processing modules is configured to generate a request message for transmission to a second one of the processing modules requesting that the second processing module transfer designated replication data to the second storage system. The request message comprises a unique message identifier that the second processing module incorporates into a subsequent response message transmitted back to the first processing module to indicate a result of the requested transfer of the designated replication data to the second storage system.

Storage system with asynchronous messaging between processing modules for data replication

An aspect of implementing bandwidth limits on replication sessions for a destination includes creating a list of routing modules having ownership of links going to the destination, calculating, by control modules, a proportionate share of a requested total bandwidth limit based on a number of slices owned and the number of routing modules. The proportionate share of the bandwidth limit is indicated by a budget value of a budget byte window (BBW). An aspect further includes prior to initiating a scan of the slices, identifying a BBW having a budget value equal or greater than the value, allocating the budget value to a thread, and reducing the budget value of the BBW by the predetermined value. An aspect also includes scanning the slices corresponding to the identified BBW, sending a number of bytes transmitted and corresponding number of pages to respective control modules resulting from the scanning, and updating corresponding BBW.

Limiting bandwidth usage on asynchronous replication sessions for a destination storage system

An apparatus in one embodiment comprises a first storage system having a plurality of storage devices. The first storage system is configured to participate in a replication process with a second storage system. The first storage system comprises a replication engine configured to request from a data module of the first storage system at least one data page to be replicated to the second storage system as part of the replication process. If the requested data page is already available in compressed form in the data module, the replication engine receives the requested data page in compressed form and transmits it to the second storage system along with at least a portion of a content-based signature of the requested data page. If the requested data page is not already available in compressed form in the data module, the replication engine receives the requested data page in uncompressed form and transmits it to the second storage system.

Opportunistic compression of replicated data in a content addressable storage system

An apparatus in one embodiment comprises a storage system having a plurality of storage devices configured to store at least a first set of data. The storage system is configured to generate a first signature for the first set of data and to verify equivalence between the first set of data and a second set of data by comparing the first signature to a second signature generated for the second set of data. The first and second sets of data comprise respective first and second sets of pages with each page having a content-based page identifier and a page address, and the first and second signatures are generated as respective functions of the page identifiers and page addresses of the respective first and second sets of pages. The first and second sets of data may comprise respective first and second storage volumes, respective first and second sets of multiple storage volumes, or other sets of data.

Signature generator for use in comparing sets of data in a content addressable storage system

An aspect of implementing bandwidth control for an asynchronous replication session includes calculating, by each of a group of control modules, a proportionate share of a requested total bandwidth limit based on a number of slices owned. The proportionate share of the bandwidth limit is indicated by a budget value via a budget byte window (BBW). An aspect further includes prior to initiating a scan of the slices by an initiator thread, identifying BBWs having a budget value equal to or exceeding a predetermined value, allocating the budget value for the BBWs to the thread, and reducing the budget value corresponding to the BBWs by the predetermined value. An aspect also includes initiating the initiator thread for slices corresponding to the identified BBWs, sending a number of bytes transmitted and corresponding number of pages to respective control modules resulting from the scanning, and updating corresponding BBWs.

Limiting bandwidth usage for an asynchronous replication session in a storage system

A storage system in one embodiment comprises a plurality of storage devices and a storage controller. The storage system is configured to operate as a source storage system in a replication process in which participation of a target storage system is simulated. In conjunction with the replication process, the source storage system is configured to monitor outbound replication messages, and responsive to a given one of the replication messages being designated for transmission over a particular communication link of a predetermined type, to generate a simulated response from the target storage system to the given replication message based at least in part on one or more simulation parameters specified for the replication process. The source storage system then processes the simulated response as if it were an actual response from the target storage system. The particular communication link of a predetermined type illustratively comprises a pseudo IP link.

Storage system with replication process utilizing simulated target responses

A system, computer program product, and computer-executable method for managing replication from a production site to a target site, wherein both the production site and the target site are content addressable data storage systems, the system, computer program product, and computer-executable method including processing a replication session from the production site to the target site, collecting information related to the replication session from the production site to the target site, analyzing the information to determine whether to change the processing of the replication session, upon a positive determination, modifying the processing of the replication session.

Growing community of inventors

Westford, MA, United States of America

William R Stronge