Nutanix, Inc.

Abhishek Gupta

Brajesh Kumar Shrivastava

Pranab Patnaik

Karan Gupta

Kai Tan

Artem Yevgenyevich Livshits

Prakash Narayanasamy

Andrey Alexseyevich Pyatkov

Angshuman Bezbaruah

Ajaykumar Rajubhai Bhammar

Naveen Kumar

Deepak Narayan

Freddy James

Ramya Uthamarajan

Siddhant Meshram

Bhoomi Sawant

Bhushan Pandit

Ranjan Mn

Akshay Pandita

Prachi Gupta

An indexing technique provides an index data structure for efficient retrieval of a snapshot from a long-term storage service (LTSS) of an archival storage system. The snapshot is generated from typed data of a logical entity, such as a virtual disk (vdisk). The data of the snapshot is replicated to a frontend data service of the LTSS sequentially and organized as one or more data objects for storage by a backend data service of LTSS in an object store of the archival storage system. Metadata associated with the snapshot (i.e., snapshot metadata) is recorded as a log and persistently stored on storage media local to the frontend data service. The snapshot metadata includes information describing the snapshot data, e.g., a logical offset range of a snapshot of the vdisk and, thus, is used to construct the index data structure. Notably, construction of the index data structure is deferred until after the entirety of the snapshot data has been replicated and received by the frontend data service.

Technique for efficiently indexing data of an archival storage system

A technique enables coordination of unrelated software components to facilitate extensive recovery point management on a snapshot or recovery point through the use of a flexible tag structure. The tag is organized and arranged as a {key=value,[value] . . . } structure wherein the key denotes an operation that requires coordination between the unrelated software components and the value(s) denote multi-cardinality that provide parameters for coordination of the operation. The multi-cardinality aspect of the flexible tag structure provides a set of values associated with the key of the tag that enables a software component and/or protocol to insert its value(s) into the tag structure for its interpretation. The technique thus provides an extensible model where multiple components/protocols use the tag to coordinate operations on the RP by conveying certain meaning/interpretations of the tag and its values.

Extensive recovery point management using tags

A technique creates a compact state of snapshot metadata and associated selected snapshots that are frequently used and maintained in memory of a node of a cluster to facilitate processing of workflow operations associated with a logical entity in a disaster recovery (DR) environment. The compact state represents a minimal subset of snapshot metadata that is frequently used to perform operations in accordance with the DR workflow operations. In addition, metadata associated with the progress of the DR workflow operations processed by the node is periodically consolidated within the compact state. Illustratively, the selected frequently used snapshots of the logical entity include (i) a recently created snapshot; (ii) one or more reference snapshots; (iii) a snapshot scheduled for replication; and (iv) any snapshot that is queued for a current or future-scheduled operation. The technique is also directed to a snapshot and metadata eviction policy that is configured to evict infrequently used snapshots and snapshot metadata to improve memory space consumption of the memory.

Technique for creating an in-memory compact state of snapshot metadata

A technique enables recovery of failover data used to generate one or more High Frequency Snapshots (HFSs) at a source and replicated to a target for storage and recovery. The target is illustratively an intermediary repository embodied as a long-term storage service (LTSS) configured to organize and store the HFSs as recovery points (RPs) in an object store. The LTSS stores a HFS identifier (ID), a logical offset in an object of the object store storing data of the HFS, and a logical timestamp associated with each replicated HFS as a key of a segment descriptor in a key-value database configured to store metadata describing the failover data of the HFS stored as one or more objects in the object store. Upon recovery of the failover data, the technique enables identification of the HFS stored in the object store and creation of a HFS index metadata structure (B+ tree) to extract the identified HFS as a RP.

Technique to store and rapidly hydrate high frequency snapshots on object storage

A framework is described that improves resource utilization during operations executing within workflows of the distributed data processing system (e.g., having a plurality of interconnected nodes) in a disaster recovery (DR) environment configured to support synchronous and asynchronous (i.e., heterogeneous) DR workflows (e.g., generating snapshots and replicating data) that include synchronous replication, asynchronous replication, nearsync (i.e., short duration snapshots of metadata) replication and migration of data objects associated with the workflows for failover (e.g., replication and/or migration) to a secondary site in the event of failure of the primary site. The framework meters (regulates) execution of the operations directed to the workloads so as to efficiently use the resources in a manner that allows timely progress (completion) of certain (e.g., high-frequency) operations and reduction in blocking (stalling) of other (e.g., low-frequency) operations by avoiding unnecessary resource hoarding/consumption and contention. Notably, the framework also provides metering and tuning of properties during execution of the workflows and maintains their state to provide for recovery.

Metering framework for improving resource utilization for a disaster recovery environment

A technique improves storage efficiency of an object store configured to maintain numerous snapshots for long-term storage in an archival storage system by efficiently determining data that is exclusively owned by an expiring snapshot to allow deletion of the expiring snapshot from the object store. The technique involves managing index data structures to enable efficient garbage collection across a very large number of data objects. When a snapshot expires, the technique obviates the need to scan the numerous snapshot data objects to determine which index structures are no longer needed and can be reclaimed (garbage collected). The technique is directed to management of underlying storage based on different sets of policies. When certain snapshots expire and are ready for deletion, the technique is directed to finding those data blocks that are no longer referenced (used) by any valid snapshots.

Garbage collection from archival of storage snapshots

A reference snapshot selection technique is configured to select a reference snapshot resolution algorithm used to determine an appropriate reference snapshot that may be employed to perform incremental snapshot replication of workload data between primary and secondary sites in a data replication environment. A reference resolution procedure is configured to process a set of constraints from the data replication environment to dynamically select the reference snapshot resolution algorithm based on a figure of merit that satisfies administrative constraints to reduce or optimize resource utilization in the data replication environment.

Dynamically adaptive technique for reference snapshot selection

Methods, systems and computer program products for high-availability computing. In a computing configuration comprising a primary node, a first backup node, and a second backup node, a particular data state is restored to the primary node from a backup snapshot at the second backup node. Firstly, a snapshot coverage gap is identified between a primary node snapshot at the primary node and the backup snapshot at the second backup node. Next, intervening snapshots at the first backup node that fills the snapshot coverage gap are identified and located. Having both the backup snapshot from the second backup node and the intervening snapshots from the first backup node, the particular data state at the primary node is restored by performing differencing operations between the primary node snapshot, the backup snapshot from the second backup node, and the intervening snapshots of the first backup node.

Computing an unbroken snapshot sequence

A system and method determining a Chain Identification Number (CID) of a source snapshot to be replicated from a source site to a target site of a virtual computing system, determining a predetermined number of potential reference snapshots based on the CID of the source snapshot, computing a closeness value between the source snapshot and each of the potential reference snapshots, and creating a list of the potential reference snapshots based on the closeness value of each of the potential reference snapshots. One snapshot from the list is selected as a reference snapshot. The source snapshot is replicated to the target site based on the reference snapshot.

Growing community of inventors

Bengaluru, India

Abhishek Gupta