Netapp, Inc.

Manan Dahyabhai Patel

Stephen Wu

Prathamesh Deshpande

Ananthan Subramanian

Wei Sun

Santhosh Selvaraj

Anil Paul Thoppil

Kevin Daniel Varghese

Arindam Banerjee

Nikul Y Patel

Meera Odugoudar

Vikhyath Rao

Raymond Jordan Go

Sanjay Subramanian Seshadri

Parag Sarfare

Szu-Wen Kuo

Asif Imtiyaz Pathan

Nikhil Mattankot

Travis Ryan Grusecki

An application may store data to a dataset comprising a plurality of volumes stored on a plurality of storage systems. The application may request a dataset image of the dataset, the dataset image comprising a volume image of each volume of the dataset. A dataset image manager operates with a plurality of volume image managers in parallel to produce the dataset image, each volume image manager executing on a storage system. The plurality of volume image managers respond by performing requested operations and sending responses to the dataset image manager in parallel. Each volume image manager on a storage system may manage and produce a volume image for each volume of the dataset stored to the storage system. If a volume image for any volume of the dataset fails, or a timeout period expires, a cleanup procedure is performed to delete any successful volume images.

Dataset image creation

Techniques are provided for implementing garbage collection and bin synchronization for a distributed storage architecture of worker nodes managing distributed storage composed of bins of blocks. As the distributed storage architecture scales out to accommodate more storage and worker nodes, garbage collection used to free unused blocks becomes unmanageable and slow. Accordingly garbage collection is improved by utilizing heuristics to dynamically speed up or down garbage collection and set sizes for subsets of a bin to process instead of the entire bin. This ensures that garbage collection does not use stale information about what blocks are in-use, and ensures garbage collection does not unduly impact client I/O processing or conversely falls behind on garbage collection. Garbage collection can be incorporated into a bin sync process to improve the efficiency of the bin sync process so that unused blocks are not needlessly copied by the bin sync process.

Garbage collection and bin synchronization for distributed storage architecture

Techniques are provided for key-value store and file system integration to optimize key value store operations. A key-value store is integrated within a file system of a node. A log structured merge tree of the key-value store may be populated with a key corresponding to a content hash of a value data item stored separate from the key. A random distribution search may be performed upon a sorted log of the log structured merge tree to identify the key for accessing the value data item. A starting location for the random distribution search is derived from key information, a log size of the sorted log, and/or a keyspace size of a keyspace associated with the key.

Key-value store and file system integration

System and method for managing and producing a dataset image across multiple storage systems

Rollback procedure for failed dataset image operation

Failover methods and systems for a networked storage environment are provided. A filtering data structure and a metadata data structure are generated before starting a replay of a log stored in a non-volatile memory of a second storage node, during a failover operation initiated in response to a failure at a first storage node. The second storage node operates as a partner node of the first storage node to mirror at the log one or more write requests received by the first storage node prior to the failure, and data associated with the one or more write requests. The filtering data structure identifies each log entry and the metadata structure stores a metadata attribute of each log entry. The filtering data structure and the metadata structure are used for providing access to a logical storage object during the log replay from the second storage node.

Network storage failover systems and associated methods

System and method for enforcing a dataset timeout for generating a dataset image

System and method for utilizing operation identifiers for communicating with storage systems to perform a dataset image operation

Creating and verifying successful creation of a dataset image of a dataset stored across a plurality of storage systems

Commit request processing for dataset image creation success

Detecting success or failure to create storage object images

Dataset image creation and failure cleanup

Growing community of inventors

Sunnyvale, CA, United States of America

Manan Dahyabhai Patel