Emc IP Holding Company LLC

Dell Products Inc.

Nitin Madan

Kalyan Chakravarthy Gunda

Srikant Viswanathan

Kedar Sadanand Godbole

Donna Barry Lewis

Bhimsen Bhanjois

Kedar Godbole

Ramprasad N Chinthekindi

Neeraj Bhutani

Abhinav Duggal

Srikanth Srinivasan

Kalidas Balakrishnan

Lan Bai

Chetan Risbud

George Mathew

Salil Dangi

Jayasekhar Konduru

Fani Jenkins

Deepa Ramesh

Gobikrishnan Sundharraj

Debasish Garai

Kalyani Sundaralingam

Vivek Nambiar

Venkata Bandlamudi

Embodiments of file restores in a Data Domain (DD) file system implementing a DD Bandwidth Optimized Open Storage Technology (DDBoost) library that translates application read and write request to DDBoost application program interfaces (APIs). A prefetch queue processor creates an intent to read the file. The application passes the file handle of the file, and the destination handle where the data must be read into. As the queue is processed, the prefetch for the request (handle/offset/length) is passed to the file server. The filesystem processes the request to open the file to load into memory. As the read request for the same file reaches the filesystem the file data is read from memory for writing to the destination handle. An extended DDBoost API expression is defined to pass the current path and destination path to the application.

File restore performance using adaptive prefetching and DDBoost application programming interfaces

Embodiments of small file restore process in deduplication file system wherein restoration requires issuing a read request within an I/O request to the file system. A prefetch queue processor creates an intent to read the file, rather than opening the file upon receiving the request. During this step, the application passes the file handle of the file, and the destination handle where the data must be read into. As the queue is processed, the prefetch for the request (handle/offset/length) is passed to the file server. The filesystem processes the request to equivalently 'open' the file, and bring the data into memory. As the read request for the same file reaches the filesystem the file data is read from memory for writing to the destination handle.

File restore performance in a deduplication file system using adaptive prefetching

Embodiments of small file restore process in deduplication file system wherein restoration requires issuing a read request within an I/O request to the file system. The process places the small files in a prefetch queue such that a combined size of the small files meets or exceeds a size of the prefetch queue as defined by a prefetch horizon. A queue processor issues a read request for the first file in the queue, scans the prefetch queue to find a read request for a file at the prefetch horizon, and prefetches the file at the prefetch horizon. The prefetch queue essentially constitutes a hint from the client that a read I/O is imminent for purposes of filling the read-ahead cache and preventing a need to issue a blocking I/O operation.

Small file restore performance in a deduplication file system

A system creates a snapshot of a part of copy management content which is displayed by a user interface, and which is selected by a user of the system as context for a subsequent message from the user. The system creates a hyperlink in the message to the snapshot which depicts the part of the copy management content. The system stores user entries for a tag that identifies at least one intended recipient of the message, a time when the message is made available for the intended recipient(s), and text for the message. The system sends the message at the time to the intended recipient(s) via an internal communication channel of the system. In response to the recipient of the message selecting the message's hyperlink to the snapshot, the system outputs the context for the message to the recipient's user interface, which enables the recipient to respond to the message.

Communicating context for messages between users in copy management systems

Embodiments of small file restore process in deduplication file system wherein restoration requires issuing a read request within an I/O request to the file system. The process places the files in a prefetch queue such that a combined size of the files meets or exceeds a size of the prefetch queue as defined by a prefetch horizon. A file handler disassociates prefetch streams from read streams. The handler prefetches the read operations and stores them in memory. The stream corresponding to a read will only open as the read requests hit the queue processor. As a result, the stream usage is very low, since the I/O, worker threads and the read streams are disassociated from each other.

File restore performance using a file handler to disassociate prefetch and read streams

A request is received to restore a file at a deduplicated storage system to a client. The file resides at the storage system as a synthetic file based on a base file at the storage system. The request includes an indication that the base file is also present at the client. Metadata generated during a backup of the file to the storage system is reviewed. The metadata includes references to data determined to be in the base file at the storage system, and references to other data determined to not be in the base file at the storage system. The other data determined to not be in the base file is read from the storage system and transmitted to the client. Upon receipt, the client assembles the requested file using the base file present at the client and the other data determined to not be in the base file.

Reducing bandwidth during synthetic restores from a deduplication file system

Replication of a filesystem or a mount point or share may replicate all data that it consists of irrespective of where the data is stored. Replication protects data irrespective of location. One method is to replicate the filesystem namespace as is while skipping the data outside of the appliance/machine so that replication cost and time are reasonable. The data outside of the machine, like cloud/tape data is protected differently. One example method includes a data protection operation configured to replication a namespace associated with multiple data tiers. During replication, data from one of the tiers is skipped while all of the namespace metadata is replicated. The recovery restores the namespace metadata and the data that was replicated from the other tier. This may be performed in connection with cyber security, for example when replicating multi-tier data to a vault.

Replication for cyber recovery for multiple tier data

Embodiments for reducing panic shutdown of components in a pipelined data processing system. Components are monitored for health, processing progress, and dependencies during normal system operation. A directed graph is generated showing non-circular dependencies of components in the pipeline. Deadlock of a particular component may or may not signal a panic condition depending on whether any of its presently downstream and depended on components are operating properly. The continuously monitored knowledge of proper operation of all downstream components is thus used to intelligently apply or defer panic alerts to keep the system operating uninterrupted from panic conditions that might soon or eventually be fixed by continued operation of the system pipeline.

Reducing false positive failure events in a health monitoring system using directional graphs

A multi-stream restore method for reducing restore time of file data to service an external read request for file data stored by a data deduplication system employs a per-file read-ahead cache that is populated by multiple parallel internal read-ahead streams. The number of parallel read-ahead streams is dynamically variable based upon processing conditions and a set of heuristically pre-determined criteria that characterize the file data including a data access pattern that is sequential and a processing load that is below a predetermined threshold. Processing conditions are periodically monitored and adjustments to the multi-stream data access are made to optimize performance.

Multi-threaded dynamic per-file read-ahead cache for deduplication system

An index associates fingerprints of file segments to container numbers of containers within which the file segments are stored. At a start of migration, a boundary is created identifying a current container number. At least a subset of file segments at a source storage tier are packed into a new container to be written to a destination storage tier. A new container number is generated for the new container. The index is updated to associate fingerprints of the at least subset of file segments to the new container number. A request is received to read a file segment. The index is queried with a fingerprint of the file segment to determine whether the request should be directed to the source or destination storage tier based on a container number of a container within which the file segment is stored.

Non-disruptive and efficient migration of data across cloud providers

Embodiments of an instant recall process and system for long-term data stored on a cloud storage tier. Embodiments include saving a content handle of a file in a cloud storage tier as an extended attribute in a single file system namespace; moving the file from the cloud storage tier to an active storage tier for data processing; recalling the file from the active storage tier to the cloud storage tier upon completion of the data processing; using the content handle from hidden metadata for a working copy of the file; and saving a hash of a segment reference as part of the extended attribute.

Instant replay of a file to a cloud tier on a deduplication file system

A method and system for storage load balancing based on virtual synthetics metadata. When storing data onto a storage cluster, data submitted thereto may often be distributed unevenly across the constituent storage nodes thereof. To address the issue, some form of load balancing (or re-distribution of data) across the storage nodes may be implemented. Existing load balancing techniques, however, tend to migrate data between storage nodes without consideration for the efficient utilization of available storage space on the storage node where the data ends up (or destination storage node). Accordingly, the disclosed method and system propose a load balancing mechanism whereby the migrated data dedupes well, thereby securing the efficient consumption of storage space on the destination storage node.

Method and system for storage load balancing based on virtual synthetics metadata

Data containers are stored at a first cloud and include file segments managed by a deduplication file system. Metadata containers are written to a log, identified by a container ID generated sequentially, and include references to data containers. An index is maintained to map segment fingerprints to container IDs. Upon starting a migration to a second cloud, a checkpoint is created identifying a container ID at a head of the log. During migration, the index is updated to map fingerprints of migrated segments to new container IDs, and referencing new data containers having the migrated segments and written to the second cloud. A request to read a file is received. The index is examined to identify a container ID associated with a data container storing segments of the file. Based on the identified container ID and the checkpoint, the data container is accessed from the first or second cloud.

Growing community of inventors

Gurugram, India

Nitin Madan