Netapp, Inc.

Apple Inc.

Network Appliance, Inc.

Advanced Micro Devices Corporation

Blue Coat Systems, Inc.

Stephen Wu

Prathamesh Deshpande

Manan Dahyabhai Patel

Szu-Wen Kuo

Parag Sarfare

Asif Imtiyaz Pathan

Nikhil Mattankot

Amber M Palekar

Mardiros Z Chakalian

Amit Borase

Sudheer Kumar Vavilapalli

John K Edwards

Sreelatha Sunkara Reddy

David A Grunwald

Sandeep Yadav

Jason Ansel Lango

Alan Stuart Driscoll

Rithin Kumar Shetty

Vikas Yadav

Brian Christopher Parkison

Robert Tsai

Prasanna Kumar Malaiyandi

Anureita Rao

David W Minnich

Muralidharan Rangachari

Manish Bhuwania

Vishal Patil

Venkatesh Babu Ks

Techniques are provided for implementing a persistent key-value store for caching client data, journaling, and/or crash recovery. The persistent key-value store may be hosted as a primary cache that provides read and write access to key-value record pairs stored within the persistent key-value store. The key-value record pairs are stored within multiple chains in the persistent key-value store. Journaling is provided for the persistent key-value store such that incoming key-value record pairs are stored within active chains, and data within frozen chains is written in a distributed manner across distributed storage of a distributed cluster of nodes. If there is a failure within the distributed cluster of nodes, then the persistent key-value store may be reconstructed and used for crash recovery.

Persistent key-value store and journaling system

An application may store data to a dataset comprising a plurality of volumes stored on a plurality of storage systems. The application may request a dataset image of the dataset, the dataset image comprising a volume image of each volume of the dataset. A dataset image manager operates with a plurality of volume image managers in parallel to produce the dataset image, each volume image manager executing on a storage system. The plurality of volume image managers respond by performing requested operations and sending responses to the dataset image manager in parallel. Each volume image manager on a storage system may manage and produce a volume image for each volume of the dataset stored to the storage system. If a volume image for any volume of the dataset fails, or a timeout period expires, a cleanup procedure is performed to delete any successful volume images.

Dataset image creation

Techniques are provided for a recovery process with selective ordering and concurrent operations in order to recover from a failure. Representations of active log structures are rebuilt within memory according to ordering values assigned to I/O operations logged within the active log structures. Representation of certain active log structures may be concurrently rebuilt based upon the active log structures comprising I/O operations that are non-overlapping within a distributed file system, have no dependencies, relate to different services, and/or target independent files. Representation of stale log structures are concurrently rebuilt within memory. While rebuilding the log structures and executing the I/O operations, a key value map is concurrently rebuilt within the memory for locating data of the I/O operations. Concurrent operations during the recovery process reduces the time to complete the recovery process, and thus reduces client downtime during the recovery process.

Recovery mechanism with selective ordering and concurrent operations

System and method for managing and producing a dataset image across multiple storage systems

Asynchronous snapshot invalidation techniques are described. According to various such techniques, an enhanced file handle structure may be defined that includes a snapshot generation ID that is to comprise a value that singularly identifies a snapshot performed at a particular point in time. In some embodiments, when a snapshot ID assigned to that snapshot is reused at a subsequent point in time, a different snapshot generation ID may be assigned to that subsequent snapshot. With respect to an in-core cache, the differing snapshot generation IDs may eliminate unacceptable ambiguity regarding respective file information sets corresponding to the initial and subsequent snapshots sharing the same snapshot ID. As a result, obsolete file information sets may be cleared from the in-core cache asynchronously, enabling improved performance. The embodiments are not limited in this context.

Techniques for asynchronous snapshot invalidation

Rollback procedure for failed dataset image operation

System and method for enforcing a dataset timeout for generating a dataset image

System and method for utilizing operation identifiers for communicating with storage systems to perform a dataset image operation

Creating and verifying successful creation of a dataset image of a dataset stored across a plurality of storage systems

Commit request processing for dataset image creation success

Detecting success or failure to create storage object images

Dataset image creation and failure cleanup

Growing community of inventors

Sacramento, CA, United States of America

Stephen Wu