Emc IP Holding Company LLC

Nathan Rosenblum

Rajiv Desai

Thomas Manville

Julio Lopez

Niraj Tolia

Vaibhav Kamra

Deepti Chheda

Diwaker Gupta

Biju Shanmugham Pillai

Girish Balvantrai Doshi

Ganesh Ghodake

Saurav Haloi

Compounding file system metadata operations via buffering is disclosed. In various embodiments, an indication to send locally-stored file system operations to a remote file system metadata server is received. A dependency graph or other data structure reflecting dependencies between the locally-stored file system operations to identify a linear chain of dependent file system operations. Operations comprising the linear chain are combined into a compound file system operation. The compound file system operation is sent to the remote file system metadata server.

Compounding file system metadata operations via buffering

Cancelling file system operations at a file system client of a distributed file system is disclosed. In various embodiments, an indication is received to send locally-stored file system operations to a remote file system metadata server. A dependency graph or other data structure reflecting dependencies between the locally-stored file system operations is used to identify a set of locally-stored file system operations the members of which are fully dominated by a create-delete pair of operations comprising the set. The locally-stored file system operations comprising the set are cancelled, without being sent to the remote file system metadata server.

File system operation cancellation

Techniques to eliminate redundant file system operations are disclosed. In various embodiments, an indication is received to send locally-stored file system operations to a remote file system metadata server. A set of related locally-stored file system operations is determined. Operations in the set are determined to satisfy a merger criteria and are merged into a single merged file system operation. The merged file system operation is sent to the remote file system metadata server.

Eliminating redundant file system operations

A combined object associated with a data chunk included in a request file is determined. An encryption key associated with the combined object and a corresponding chunk hash value associated with the data chunk are used to determine a corresponding chunk key associated with the data chunk. At least a locator to be used to retrieve the combined object and the corresponding chunk key associated with the data chunk are provided to a requesting system.

Secure cloud-based storage of data shared across file system objects and clients

Providing disaster recovery of a distribute file system metadata server as a service is disclosed. In various embodiments, an indication is received to provide alternative access to a file system metadata server. A copy of a self-describing backup of the file system metadata server is obtained, e.g., from cloud-based storage. The backup is used to provide access to a read only instance of the file system metadata server.

Disaster recovery as a service

Techniques to provide a de-duplicating distributed file system using a cloud-based object store are disclosed. In various embodiments, a request to store a file comprising a plurality of chunks of file data is received. A determination to store at least a subset of the plurality of chunks is made. The request is responded to at least in part by providing an indication to store two or more chunks comprising the at least a subset of the plurality of chunks comprising the file as a single stored object that includes the combined chunk data of said two or more chunks.

De-duplicating distributed file system using cloud-based object store

A method, article of manufacture, and apparatus for deduplicating data on a distributed file system is discussed. Metadata requests are made to an object store to determine whether the store already contains a data object. If the object store does contain the data object, a commit request is sent to a metadata server. If the object store does not contain the data object, the data object is transmitted to the object store.

Data deduplication on a distributed file system

Techniques to provide secure cloud-based storage of data shared across file system objects and clients are disclosed. In various embodiments, a primary encryption key is determined for an object associated with a plurality of component chunks of file system data. The primary encryption key is used to generate for each of said component chunks a corresponding chunk key, based at least in part on the primary encryption key and data comprising or otherwise associated with the chunk. The respective chunk keys are provided to a file system client configured to create and store the object at least in part by encrypting each chunk included in the plurality of component chunks using the chunk key provided for that chunk to generated encrypted chunk data, and combining the encrypted chunk data to create and store the object.

A multilayer manifest enables any file written to an object store using a proprietary storage process, including large virtual machine backup images, to be recovered without being dependent on the proprietary storage process. Upon synchronizing a file written to an object store in cloud-enabled storage, a multilayer manifest process generates a family of manifest files in a self-contained format that represents all the necessary information required to reconstruct the file. The family of manifest files includes an ancestral record used to initiate retrieval of a file, and multiple layers of manifest files associated with the ancestral record. During retrieval, each of the one or more multilayer manifest files is successively reconstructed and used to securely retrieve the next manifest file until no further manifest files remain in the family. Reconstruction of the last multilayer manifest file results in the recovery of the file for which the manifest files were generated.

Growing community of inventors

San Francisco, CA, United States of America

Nathan Rosenblum