International Business Machines Corporation

Seagate Technology Incorporated

Kenneth F Day, Iii

John Jay Wolfgang

Kenneth Wayne Boyd

Philip Matthew Doatmas

Douglas William Dewey

Kirby Grant Dahman

Ian Davies

Michael Edward Factor

Robert Bruce Nicholson

Michael David Barrell

Alfredo M Diaz

Harry M Yudenfriend

Aviad Zlotnick

Carlos Francisco Fuente

William James Scales

James Lee Hafner

Ying Chen

Shachar Fienblit

Bret S Weber

Orit Nissan-Messing

Lee Jason Sanders

Thomas Charles Jarvis

Daniel Alexander Ford

Wendy Ann Belluomini

Henry Esmond Butterworth

Olympia Gluck

David Allan Pease

Tarek Makansi

John Earle Lindley

Amiram Hayardeny

Stephen F Edel

John E Bish

Edward R Morse

William D Lamear, Jr

William D Burke

Michael A Truty

Eddie Lin

Bill J Seales

Systems and methods are disclosed for multiple duration deduplication entries. A processor may be configured to perform data deduplication operations, including setting a first indicator for first metadata stored to a hash database identifying a first time interval to maintain the first metadata in the database, and setting a second indicator for second metadata stored to the database identifying a second time interval to maintain the second metadata in the database longer than the first time interval. The processor may remove the first metadata from the database when no duplicative data is detected for the first metadata within the first time interval, and remove the second metadata from the database when no duplicative data is detected for the second metadata within the second time interval.

Multiple duration deduplication entries

Method and apparatus for managing data in a distributed data storage system, such as but not limited to a cloud computing environment. In some embodiments, snapshots of a data set are uploaded from a source storage subsystem to a cloud store, along with intervening difference information volumes (DIVs). The DIVs are data structures that identify a set of updated data blocks that have been changed by the source storage subsystem between each successive pair of the snapshots. A reader subsystem requests and uses the latest DIV to request the latest set of changed data blocks from the cloud store, and uses the latest set of changed data blocks to update a previous snapshot to generate a copy of the most recent snapshot. The source and reader subsystems can comprise Internet of Things (IoT) devices, client devices, edge computing devices, etc. from different vendors and which utilize different protocols.

Delta information volumes to enable chained replication of data by uploading snapshots of data to cloud

An apparatus may include a controller configured search a hash database storing entries corresponding to hash values of previously stored data to find a hash page of the hash database corresponding to a range of hash values including the first hash value. When the hash page is found, the controller may be configured to determine whether the hash page does not include an entry for the first hash value, the first hash page further including a base hash value for the range of hash values. When the hash page does not include an entry for the first hash value, the controller may be configured to generate a first entry of the first hash page for the first hash value, the first entry including an offset value from the base hash value corresponding to the first hash value and a data location at which the received data is to be stored.

Deduplication database management

An apparatus for controlling a storage system having a data replication function, comprises: a storage array component being operable to send notification to a replication engine that a write of data to a primary storage location by a host is subject to data replication; the replication engine being operable to receive the notification and in response to instruct the storage array to copy the data to a secondary storage location; wherein the data is copied to the secondary storage location unmediated by the replication engine.

Apparatus and method for managing storage copy services systems

Methods and apparatus are provided for copying data from a primary storage facility to a secondary storage facility which reduce the workload on the storage controller in the primary facility and minimize bandwidth usage. The primary storage facility includes a primary data replication appliance which transfers data to a secondary replication appliance. Updated data from a host is both stored through a storage controller in the primary facility and also received by the primary replication appliance. Logic in the primary replication appliance determines whether the immediately previous version of the data is in a buffer from a previous storage operation. If so, the current (updated) version of the data is compared with the previous version and the difference, such as calculated through a bit-wise exclusive-OR operation, is transferred to the secondary replication appliance. The process is reversed in the secondary replication appliance and the recreated updated version of the data stored in the secondary facility.

Transferring data from a primary data replication appliance in a primary data facility to a secondary data replication appliance in a secondary data facility

An apparatus, system, and method are disclosed for a read-before-write storage controller instruction. A sequencer receives an atomic read-before-write instruction comprising new data, a target address for the new data, and an undo log address. An I/O unit reads old data from the target address, writes the old data and the target address to the undo log address, and writes the new data to the target address as directed by the sequencer.

Apparatus, system, and method for a read-before-write storage controller instruction

A mechanism is provided for managing operations in a data storage system. A first storage controller monitors operation state information associated with a plurality of operation states stored in a data structure. The first storage controller detects a change in the operation state information. The change in the operation state information is in response to a managing entity updating the operation state information to a first operation state in the plurality of operation states in the data structure. Responsive to detecting the change in the operation state information, the first storage controller identifies a reference to a first operation associated with the first operation state updated by the managing entity. The first operation is one operation in a plurality of operations. The first storage controller performs the first operation associated with the first operation state.

Idempotent storage replication management

An apparatus, system, and method are disclosed for executing functions in an event driven environment. A bridge module stores parameters and a proxy function pointer for a proxy function in a bridge structure for a co-routine context executing in a co-routine environment. The proxy function is configured to execute in an event driven environment. The bridge module also stores a bridge pointer for the bridge structure in a bridge pointer register of an event driven context structure. A swap context module swaps to the event driven context. An execute module retrieves the proxy function pointer from the bridge structure using the bridge pointer if the bridge pointer register is not null, executes the proxy function, and stores the results of the proxy function in the bridge structure.

Apparatus, system, and method for executing functions in an event driven environment from a co-routine environment

A data storage method comprises storing first data in at least a first data chunk, wherein the first data chunk is a logical representation of one or more sectors on at least a first disk drive in a storage system; storing first metadata, associated with the first data, in at least a first appendix, wherein the first appendix is a logical representation of a sector region on at least the first disk drive in the storage system, and wherein the first metadata comprises first atomicity metadata (AMD) and first validity metadata (VMD) associated with the first data; and storing a copy of the first VMD for the first data in at least one low latency non-volatile storage (LLNVS) device.

Data integrity validation in storage systems

An apparatus, system, and method are disclosed for switching a volume address association in a point-in-time copy relationship. A copy module creates a point-in-time copy structure of a source volume at a target volume. A migration module copies data from the source volume to the target volume. A detection module detects data corruption in the source volume. A switch module switches a logical volume address from a source volume address of the source volume to a target volume address of the target volume in response to detecting the data corruption, redirecting data reads and writes to the target volume. In one embodiment, a tracking module tracks a current target volume copy of source volume data and redirects an access for source volume data without the current target volume copy to the source volume.

Apparatus, system, and method for switching a volume address association in a point-in-time copy relationship

An apparatus and method to replicate one or more files between non-symmetric storage systems are disclosed. The method supplies a first storage system comprising a first configuration, a first volume, a first file system, and a first replication appliance comprising first replication appliance memory. The method further supplies a second storage system comprising a second configuration, a second volume, a second file system, and a second replication appliance comprising second replication appliance memory, where the first configuration differs from the second configuration. The first storage system receives a dataset, writes that dataset to the first volume as a first file. Applicants' method then replicates the first file to the second volume as a second file, and maintains in the second replication appliance memory a second replication appliance mapping associating the first file with the second file.

Apparatus and method for file-level replication between two or more non-symmetric storage sites

A method to optimize the transmission of data from (N) primary backup appliances interconnected to a plurality of second backup appliances by a single communication link, wherein (N) is greater than 1, by transferring a data set to one or more secondary backup appliances by two or more of the (N) primary backup appliances using the communication link, and completing those transfers of the data sets by the two or more primary backup appliances at the same time.

Apparatus and method to adjust data transfer rate

In a data processing environment, data is replicated to a remote or secondary storage device in a manner which reduces the adverse performance effects and inefficient bandwidth usage imposed by the conventional one-transaction-at-a-time process. Transactions to be transferred are grouped by a replication manager by selecting transactions having start times earlier than the completion time of a first transaction. Thus, no transaction in a group will be dependent upon any other transaction in the group. Once selected, all transactions in the group may then be transferred to the secondary storage device.

Asynchronous replication of data

A method to select a captain control node from a plurality of interconnected control nodes is disclosed. The method provides by each of the plurality of control nodes a first signal to each of the other control nodes, and then receives by each of the plurality of control nodes, a response signal from each of the other control nodes. The method then calculates by each of the plurality of control nodes individual response times for each of the other control nodes, and determines by each control node its aggregate response time. The method then determines whether to select a captain control node using the aggregate response times. If the captain control node is selected using the aggregate response times, then the method then determines the minimum aggregate response time, and designates the control node having that minimum aggregate response time the captain control node.

Apparatus and method to select a captain from a plurality of control nodes

A method is provided to improve the efficiency of the transmission of consistency groups across multiple storage components by ensuring that volume transactions are evenly allocated among backup components. Each volume is initially assigned to a specified backup component. Once a color period begins, volume transactions are transmitted to the assigned backup component. Each backup component accumulates data transfer information for its assigned volumes throughout at least a portion of the color period and calculates a new volume assignment for the next color period. Before the start of the next color period, a captain storage controller transmits any new assignments to the backup components. During a next color period, the process is repeated using the new assignments.

Growing community of inventors

Tucson, AZ, United States of America

Kenneth F Day, Iii