Emc Corporation

Emc IP Holding Company LLC

Cisco Technology, Inc.

Junxu Li

Windsor Hsu

Scott Auchmoody

Mark D Malamut

Adam E Brenner

Jeffrey Norton

Jennifer Starling

Erik Shawn Hansen

Vishwas Srinivasan

Rajesh Rajashekar

Yuping Li

In one embodiment, systems and methods for performing asynchronous local migration of metadata between data stores and asynchronous remote replication of metadata between sites are described. The methods may use various configurations, including 1-to-1, 1-to-N, N-to-1, M-to-N, etc. The method for performing asynchronous local migration at a first site may include pausing critical operation(s) at an old data store, copying metadata from the old data store to a new data store, flagging table(s) in the old data store as complete, and deleting the metadata from the old data store. The method for asynchronous remote replication may include determining that local migration is complete, identifying second metadata from the new data store for which the first site is a primary authority, sending, to the second site, the second metadata, receiving, from the second site, third metadata for which the second site is the primary authority and storing the third metadata.

Asynchronous metadata replication and migration between compute sites

One embodiment is related to a method for performing garbage collection at a backup system, comprising: determining an object that is present in a preceding backup but is no longer present in a present backup when or after the present backup is made; associating an indicator indicative of the preceding backup with the object in response to the determination that the object is no longer present in the present backup; determining whether the object is to be deleted from the backup system based on the indicator and a policy during garbage collection; and deleting the object in response to determining that the object is to be deleted.

Garbage collection: timestamp entries and remove reference counts

Embodiments are described for presenting a backed up data object to a client, in a native file format, in response to a request from the client to retrieve the requested data object. The request includes security attributes of the requested data object. If the security attributes in the request match the security attributes stored with the backed up data object, then the requested data objet is returned to the requesting client. When a backup is performed, metadata associated with each backed up data object is stored in the backup data set. When a request is received to restore a backed up data object, a metadata tree is constructed that represents the directory structure of the backed up data objects, and further includes security attributes for each data object and an offset into the backup data set where the requested data object can be found.

Extended snapshot using backup and microservice

A recipe is stored for an application where the recipe specifies parameters and actions for restoring, shutting down, and backing up the application. A snapshot backup of the application is copied from a backup system to a recovery system where the application is restored according to the recipe. After the application is restored, the application is shutdown. While the application is shutdown, the application is backed up to create an application-consistent backup of the application.

Primed application recovery

Backup objects, native security descriptors that describe access rights to the backup objects, and object metadata are received from platforms where each backup object has corresponding object metadata. Each native security descriptor is transformed into a platform independent security descriptor. A partitioned structure is generated, including by associating each platform independent security descriptor with those object metadata that correspond to the plurality of backup objects for which that platform independent security descriptor describes access rights to.

Secure search

According to one embodiment, a first storage system receives a first data stream from a second storage system over a network. The first data stream includes data objects and differential object information identifying at least one data object missing from the first data stream. A difference between the first data stream and a second data stream that has been previously received is determined based on the differential object information, including identifying a data object that has been added, deleted, or modified in view of the second data stream. The first data stream is reconstructed based on the second data stream and the difference between the first data stream and the second data stream, generating a third data stream. The third data stream is stored in a persistent storage device of the first storage system, the third data stream representing a complete first data stream without a missing data object.

Method and system for accelerating data movement using change information concerning difference between current and previous data movements

Systems and methods are described for backing up files and directories using a common backup format. The files and directories may be represented by objects within a data stream constructed using the common backup format. The data streams may be traversed and updated using a size tree such that modifications are made to individual objects within the data streams without complete traversal. This process results in efficient management of storage systems as read and write operations are not dependent on exhaustive traversal of data streams.

Common backup format and log based virtual full construction

Data chunks encrypted using an encryption key are backed up to a server. Each chunk is associated with plain and encryption signatures. The plain signature is based on an unencrypted version of a chunk. The encryption signature is based on an encrypted version of the chunk. A new data chunk is identified and a new plain signature for the new chunk is calculated. A request is made for a current key and the new chunk is encrypted using the current key to obtain a new encryption signature. The new encryption and plain signatures are sent to the server for comparison against the existing encryption and plain signatures. If the new encryption signature does not match an encryption signature of an existing chunk and the new plain signature matches a plain signature of the existing chunk, the new chunk is transmitted to the server to replace the existing chunk.

Secure data deduplication

Techniques for determining optimal time window for data movement from a source storage system to a target storage system are described herein. According to one embodiment, statistics data is received representing historic performance statistics over a predetermined period of time by a source storage system, where the historic performance statistics include resource consumption of a plurality of resources including at least one of a processor, memory, input-output (IO) transactions, and network bandwidth. An analysis is performed by an analysis module executed by a processor on the historic performance statistics to determine an optimal time window within the predetermined time period for data movement from the source storage system to a target storage system based on the analysis. A scheduler executed by the processor is to schedule the data movement from the source storage system to the target storage system according to the optimal time window.

Method and system for determining optimal time period for data movement from source storage to target storage

Techniques for deduplicating a backup stream with ACL data embedded therein are described. According to one embodiment, a storage system receives a backup stream from a client, where the backup stream includes multiple data streams to be stored in the storage system. Each data stream represents a file or a directory of one or more files of a file system associated with the client. The storage system is to scan the backup stream to recognize an access control list (ACL) marker associated with at least one of the data streams, where the ACL marker identifies ACL data representing an ACL of the at least one data stream. The storage system is to chunk using a predetermined chunking algorithm the ACL data into a plurality of ACL data chunks to be stored in the storage system in a deduplicated manner.

Special markers to optimize access control list (ACL) data for deduplication

Techniques for deduplicating a data stream based on boundary markers embedded therein are described. According to one embodiment, a data stream is received from a client having a sequence of a plurality of data objects, where to data stream represents a file or a directory of one or more files of a file system associated with the client. In response, the data stream is deduplicated into a plurality of deduplicated chunks in view of boundaries of the data objects.

Method and system for handling object boundaries of a data stream to optimize deduplication

Techniques for deduplicating a data stream with checksum data embedded therein are described. According to one embodiment, a first data stream is received from a client having a plurality of data regions and a plurality of checksums for verifying integrity of the data regions embedded therein, where the first data stream represents a file or a directory of one or more files of a file system associated with the client. In response the first data stream with the checksums removed is deduplicated into a plurality of deduplicated chunks.

Method and system for processing checksum of a data stream to optimize deduplication

Techniques for accelerating data movement from a source storage system to a target storage system are described herein. According to one embodiment, a first data stream is received to be transmitted to a remote storage system over a network, the first data stream having multiple data objects, each identified by a data object identifier (ID). A difference is determined between the first data stream and a second data stream that has been previously transmitted to the remote storage system, including identifying one or more data objects that are added, deleted, and/or modified. Differential object information is transmitted to the target storage representing the difference between the first data stream and the second data stream, as well as one or more data objects that are missing in the second data stream without having to transmit all data objects of the first data stream.

Growing community of inventors

Pleasanton, CA, United States of America

Junxu Li