Nutanix, Inc.

Rishi Bhardwaj

Mohit Aron

Karan Gupta

Venkata Ranga Radhanikanth Guturi

Pavan Kumar Konka

Dheeraj Pandey

Ajeet Singh

Brent Chun

Tabrez Parvez Memon

Rohit Jain

Parthasarathy Ramachandran

Vanita Prabhu

Gowtham Alluri

Kamalneet Singh

Amod Vilas Jaltade

Aditya Vilas Jaltade

Niranjan Sanjiv Pendharkar

Alok Nemchand Kataria

Roger Sean Liao

Vinayak Hindurao Khot

Shubham Shukla

Pete Wyckoff

Biswa Ranjan Panda

Chinmay Dinesh Kamat

Aashray Arora

Anand Varma Chekuri

Arumugam Arumugam

An improved architecture is provided which enables significant convergence of the components of a system to implement virtualization. The infrastructure is VM-aware, and permits scaled out converged storage provisioning to allow storage on a per-VM basis, while identifying I/O coming from each VM. The current approach can scale out from a few nodes to a large number of nodes. In addition, the inventive approach has ground-up integration with all types of storage, including solid-state drives. The architecture of the invention provides high availability against any type of failure, including disk or node failures. In addition, the invention provides high performance by making I/O access local, leveraging solid-state drives and employing a series of patent-pending performance optimizations.

Architecture for implementing a virtualization environment and appliance

Various embodiments set forth a computer-readable media storing program instructions that, when executed by one or more processors, cause the processors to perform steps of establishing, by a first node of a first cluster, a respective first connection with each of a plurality of second nodes in a second cluster; and establishing, by third node of the first cluster, a second connection with the first node. The third node is prevented from establishing a connection with any of the second nodes. The steps further include sending, by the third node, a message to the second cluster by sending the message to the first node via the second connection, where the first node is configured to forward the message to one of the second nodes via a corresponding one of the respective first connections.

Inter-cluster communication

An illustrative embodiment disclosed herein is an apparatus including a processor having programmed instructions to update a versioned object by appending a latest version of the versioned object to a data structure, read a predetermined version identifier corresponding to the latest version from a pointer entry, and look up the latest version of the versioned object, in the data structure, using the predetermined version identifier corresponding to the latest version of the versioned object.

System and method for supporting versioned objects

Methods, systems and computer program products for computer system data management. A leader node that is in a leader-follower relationship with a follower node is deployed to manage metadata across a plurality of interconnected computing nodes of the computer system. Upon a failure of any sort that results in a loss of operation or a disconnection of the leader node, the followers undertake steps to bring up a new leader. Such steps to bring up a new leader include electing a new leader to replace the lost or disconnected leader and initializing the data state of the new leader node by causing the leader to request incremental snapshots from followers. Additional metadata that might have changed since closing the incremental snapshots can also be retrieved from the followers. The metadata that might have changed since the time of closing the incremental snapshot is subjected to verification using a consensus algorithm.

Managing incremental snapshots for fast leader node bring-up

Various embodiments set forth techniques for managing and/or accessing metadata associated with a vblock, systems implementing said techniques, and computer-readable media storing instructions for performing said techniques. In some embodiments, one or more computer-readable media store instructions that, when executed by one or more processors, cause the one or more processors to perform steps including receiving a request for metadata associated with a vblock; accessing a merged metadata record associated with the vblock, where the merged metadata record comprises metadata corresponding to metadata in metadata records for all but a last snapshot or a live vblock having a metadata record, and a first identifier of the last snapshot or the live vblock having a metadata record; and returning the requested metadata based on the metadata in the merged metadata record and metadata in the metadata record identified by the first identifier.

Vblock metadata management

Systems for high-performance distributed computing. The systems include techniques for managing data and metadata across multiple nodes. A method embodiment commences by storing data at a node using a first storage mechanism that is local to the node. A first set of metadata is configured to identify a storage location for the stored data. The first set of metadata is stored using the same first storage mechanism that is local to the node. For accessing the first set of metadata, a second set of metadata is configured to identify a storage location for the first set of metadata. The second set of metadata is stored using a second storage mechanism that comprises a distributed metadata storage facility that stores metadata across multiple storage locations having at least one of the multiple storage locations that is not local to the node that stores data and metadata using the first storage mechanism.

Efficient metadata management

Various embodiments set forth techniques for managing metadata associated with a vblock, In some embodiments, one or more computer-readable media store instructions that, when executed by one or more processors, cause the one or more processors to perform steps including receiving a request to write data to a live vblock, wherein the request to write data is a first write request for the live vblock; accessing a merged metadata record associated with the live vblock, wherein the merged metadata record comprises metadata corresponding to metadata in metadata records for all but a last snapshot included in a set of snapshots having a metadata record; adding metadata associated with the request to write data to a metadata record for the live vblock; merging a metadata record for the last snapshot into the merged metadata record; and updating a first identifier of the merged metadata record to identify the live vblock.

A technique replicates an index of an operations log (oplog) from a primary node to a secondary node of a cluster in the event of failure. The oplog functions as a staging area to coalesce random write operations directed to a virtual disk (vdisk) stored on a backend storage tier. The oplog temporarily caches write data as well as metadata describing the write data. The metadata includes descriptors to the write data corresponding to offset ranges of the vdisk and are used to identify ranges of write data for the vdisk that are cached in the oplog. To facilitate fast lookup operations of whether write data is cached in the oplog, an oplog index provides a state of the latest data for offset ranges of the vdisk that enables fast failover of metadata used to construct the oplog index in memory without downtime or significant metadata replay.

Technique for replicating oplog index among nodes of a cluster

Systems for low-latency data access in distributed computing systems. A method embodiment commences upon generating a first storage area in local storage of a first computing node. Access to the first storage area is provided through the first computing node. A second storage area is generated wherein the second storage area comprises a first set of metadata that comprises local storage device locations of at least some of the local storage areas of the first storage area. A set of physical access locations of the second storage area is stored to a database that manages updates to the second set of metadata pertaining to the second storage area. Accesses to the first storage area are accomplished by querying the database to retrieve a location of the second set of metadata, and then accessing the first storage area through one or more additional levels of metadata that are node-wise collocated.

Low latency access to physical storage locations by implementing multiple levels of metadata

Systems for low-latency data access in distributed computing systems. A method embodiment commences upon generating a first storage area in local storage of a first computing node. Access to the first storage area is provided through the first computing node. A second storage area is generated wherein the second storage area comprises a first set of metadata that comprises local storage device locations of at least some of the local storage areas of the first storage area. A set of physical access locations of the second storage area is stored to a database that manages updates to the second set of metadata pertaining to the second storage area. Accesses to the first storage area are accomplished by querying the database retrieve a location of the second set of metadata, and then accessing the first storage area through one or more additional levels of metadata that are node-wise collocated.

A system using pulsed leader consensus management that can manage storage by electing a leader node from a group of nodes and fulfilling read requests for the group of nodes if a sync state is not expired. The sync state can correspond to two conditions: first, that the nodes in the group direct the write requests for the group to the leader node; second, that a health pulse is detected, wherein the health pulse is generated if the leader node is alive (e.g. has not failed, terminated, or become non-responsive). If either of the two conditions fail, the sync state can be considered expired and the group of nodes may fall back to consensus-based read mechanisms, such as Paxos, until a new leader node is elected.

Growing community of inventors

San Jose, CA, United States of America

Rishi Bhardwaj