Dell Products Inc.

Kedar Godbole

Donna Barry Lewis

Nitin Madan

Srikant Viswanathan

Bhimsen Bhanjois

Sandeep Nirmale

Rajendra Kumar Bhairy Raj

Improving read performance using pre-fetches by encoding an extent map identifying valid data of the file as extents that may be interspersed with holes constituting invalid data, and storing the extent map in the client. The extent map generates pre-fetches of data that land on only extents of the file and avoids the holes, by one of: transferring, upon the read request, the extent map to a filesystem storing the file, and bringing the extents from the extent map into memory ahead of the read request; or using, in the network client, the extent map to generate prefetch requests and execute read-ahead operations on the filesystem. The method may also apply where the application only wants to read what has changed since the last backup. In this case, the application has a map of what has changed, and can choose to read just the right extents.

Extent restore performance using smart prefetching

Disclosed is a storage system comprising: receiving a first data segment and first metadata associated with the first data segment to be stored in the storage system; storing the first data segment and the first metadata in a persistent storage device of the storage system; compressing the first data segment using a predetermined compression algorithm to generate a first compressed data segment; and storing the first metadata and the first compressed data segment in a solid state drive (SSD) cache device of the storage system, including aligning the first metadata and the first compressed data segment to a page boundary of the SSD device to reduce a number of input and output (IO) operations required for accessing the first metadata and the first compressed data segment from the SSD cache device.

Method to opportunistically reduce the number of SSD IOs, and reduce the encryption payload, in an SSD based cache in a deduplication file system

During a current backup of a file system of a client, a determination is that a file in the current backup includes first data regions found in a file from a previous backup, and second data regions that are new. A virtual synthetic file is generated by writing the second data regions, and synthesizing the first data regions by referencing the file from the previous backup. A request received to restore the particular file includes identifications of other files still present at client. A determination is that the particular file corresponds to the virtual synthetic file and that the other files still present at client include the file from the previous backup. The second data regions are read to send to client. A complete version of the particular file is assembled at client using the file from the previous backup, still present at client, and the second data regions.

Reducing bandwidth during synthetic restores from a deduplication file system

A prefetch mechanism stores data in the memory allocated in a remote passive (remote) node in a High Availability active-passive node pair. A read-ahead cache that can host different parts of a file is created on both the active and the passive nodes. The system improves the performance of sequential restores by using memory available on a passive node to supplement the local (active node) memory to hold the prefetched data for the common case of large sequential restores. This extra memory overcomes issues when a current prefetch fails to keep up with the target restore performance. Filesystem redirection is thus accomplished by servicing an I/O request by sending it another node that itself caches the data.

Sequential file restore performance using filesystem redirection

Embodiments of a system and method to track the locality of a file being restored at the time of prefetching; and a mechanism to dynamically adjust the prefetching parallelism, per read batch, optimally based on the locality and other heuristics, such as system load. A process tracks locality of data elements in a batched data stream, as corresponds to a number of different container IDs accessed by the data elements. The prefetch nominally works serially on the data elements, however, if the locality exceeds a threshold separating acceptable versus non-acceptable distribution of data accesses, each batch is divided into a number of smaller sub-batches that are then pre-fetched in parallel with one another.

Sequential file restore performance using dynamic prefetch based on file locality

Embodiments of file restores in a Data Domain (DD) file system implementing a DD Bandwidth Optimized Open Storage Technology (DDBoost) library that translates application read and write request to DDBoost application program interfaces (APIs). A prefetch queue processor creates an intent to read the file. The application passes the file handle of the file, and the destination handle where the data must be read into. As the queue is processed, the prefetch for the request (handle/offset/length) is passed to the file server. The filesystem processes the request to open the file to load into memory. As the read request for the same file reaches the filesystem the file data is read from memory for writing to the destination handle. An extended DDBoost API expression is defined to pass the current path and destination path to the application.

File restore performance using adaptive prefetching and DDBoost application programming interfaces

Embodiments of small file restore process in deduplication file system wherein restoration requires issuing a read request within an I/O request to the file system. A prefetch queue processor creates an intent to read the file, rather than opening the file upon receiving the request. During this step, the application passes the file handle of the file, and the destination handle where the data must be read into. As the queue is processed, the prefetch for the request (handle/offset/length) is passed to the file server. The filesystem processes the request to equivalently 'open' the file, and bring the data into memory. As the read request for the same file reaches the filesystem the file data is read from memory for writing to the destination handle.

File restore performance in a deduplication file system using adaptive prefetching

Embodiments of small file restore process in deduplication file system wherein restoration requires issuing a read request within an I/O request to the file system. The process places the small files in a prefetch queue such that a combined size of the small files meets or exceeds a size of the prefetch queue as defined by a prefetch horizon. A queue processor issues a read request for the first file in the queue, scans the prefetch queue to find a read request for a file at the prefetch horizon, and prefetches the file at the prefetch horizon. The prefetch queue essentially constitutes a hint from the client that a read I/O is imminent for purposes of filling the read-ahead cache and preventing a need to issue a blocking I/O operation.

Small file restore performance in a deduplication file system

Embodiments of small file restore process in deduplication file system wherein restoration requires issuing a read request within an I/O request to the file system. The process places the files in a prefetch queue such that a combined size of the files meets or exceeds a size of the prefetch queue as defined by a prefetch horizon. A file handler disassociates prefetch streams from read streams. The handler prefetches the read operations and stores them in memory. The stream corresponding to a read will only open as the read requests hit the queue processor. As a result, the stream usage is very low, since the I/O, worker threads and the read streams are disassociated from each other.

File restore performance using a file handler to disassociate prefetch and read streams

A request is received to restore a file at a deduplicated storage system to a client. The file resides at the storage system as a synthetic file based on a base file at the storage system. The request includes an indication that the base file is also present at the client. Metadata generated during a backup of the file to the storage system is reviewed. The metadata includes references to data determined to be in the base file at the storage system, and references to other data determined to not be in the base file at the storage system. The other data determined to not be in the base file is read from the storage system and transmitted to the client. Upon receipt, the client assembles the requested file using the base file present at the client and the other data determined to not be in the base file.

Growing community of inventors

Pune, India

Kedar Godbole