Nutanix, Inc.

Tabrez Parvez Memon

Miao Cui

Binny Sher Gill

Rishi Bhardwaj

Rohit Jain

Karan Gupta

Manosiz Bhattacharyya

Vinayak Hindurao Khot

Jaya Singhvi

Harshit Agarwal

Niranjan Sanjiv Pendharkar

Alok Nemchand Kataria

Pavan Kumar Konka

Kannan Muthukkaruppan

Shubham Shukla

Pete Wyckoff

Brian Byrne

Felipe Franciosi

Gregory A Smith

Aditya Ramesh

Kamalneet Singh

Sreejith Mohanan

Ashwin Thennaram Vakkayil

Amod Vilas Jaltade

Pradeep Kashyap Ramaswamy

Biswa Ranjan Panda

Gregory Andrew Smith

One embodiment of the present invention provides a system that can manage access to a service from a cluster of computing nodes. An instance of the system can operate on a respective computing node. During operation, the system instance can identify an Internet Protocol (IP) address, which provides access to the service from the cluster of computing nodes to a client device at a client site. The system instance can select the computing node for hosting the IP address in the cluster of computing nodes based on a set of selection criteria. The selection is performed independently at the computing node. The system instance can then assign the IP address to the computing node. The assignment allows a request for the service to be directed to the computing node. Subsequently, the system instance can facilitate the service from the cluster of computing nodes based on the request.

Efficient virtual IP address management for service clusters

A technique improves implementation of an index for an operations log (oplog) that coalesces random write operations directed to a virtual disk (vdisk) stored on an extent store. The oplog temporarily caches data associated with the random write operations (i.e., write data) as well as metadata describing the write data. The metadata includes descriptors to the write data stored on virtual address regions, i.e., offset ranges, of the vdisk and are used to identify the offset ranges of write data for the vdisk that are cached in the oplog. To facilitate fast lookup operations of the offset ranges when determining whether write data is cached in the oplog, an oplog index provides a state of the latest data for offset ranges of the vdisk. The technique improves implementation of the oplog index by storing the oplog index in storage class memory, such as persistent memory, to obviate failure and subsequent recovery of the oplog index.

Technique for improving operations log indexing

Various embodiments set forth techniques for maintaining replicas of a data set. The techniques include storing, by a first node, a first replica of the data set on the first node; storing, by a second node, a second replica of the data set on the second node; and performing, by the first node and based on first metadata associated with the first replica, a first data management activity on the first replica that changes a first format of the first replica so that the first format of the first replica is different from a second format of the second replica.

Independent encoding and transformation of related data replicas and localized background data management in a distributed file system

Systems for high-performance distributed computing. The systems include techniques for managing data and metadata across multiple nodes. A method embodiment commences by storing data at a node using a first storage mechanism that is local to the node. A first set of metadata is configured to identify a storage location for the stored data. The first set of metadata is stored using the same first storage mechanism that is local to the node. For accessing the first set of metadata, a second set of metadata is configured to identify a storage location for the first set of metadata. The second set of metadata is stored using a second storage mechanism that comprises a distributed metadata storage facility that stores metadata across multiple storage locations having at least one of the multiple storage locations that is not local to the node that stores data and metadata using the first storage mechanism.

Efficient metadata management

Various embodiments set forth techniques for transactional allocation and deallocation of blocks in a block store. A first technique includes sending a first request that causes a non-persistent allocation of a block. The first technique also includes adding a first entry in a log recording the allocation as tentative, sending a second request that causes persistence of the allocation, and adding a second entry in a log recording the allocation as finalized. A second technique includes adding a first entry in a log recording a deallocation of a block, sending a first request that causes the deallocation of the block and causes the block to be unavailable for reallocation in a non-persistent manner, adding a second entry in the log recording that the deallocation is finalized, and sending a second request that causes the block to be made available for reallocation.

Transactional allocation and deallocation of blocks in a block store

A technique replicates an index of an operations log (oplog) from a primary node to a secondary node of a cluster in the event of failure. The oplog functions as a staging area to coalesce random write operations directed to a virtual disk (vdisk) stored on a backend storage tier. The oplog temporarily caches write data as well as metadata describing the write data. The metadata includes descriptors to the write data corresponding to offset ranges of the vdisk and are used to identify ranges of write data for the vdisk that are cached in the oplog. To facilitate fast lookup operations of whether write data is cached in the oplog, an oplog index provides a state of the latest data for offset ranges of the vdisk that enables fast failover of metadata used to construct the oplog index in memory without downtime or significant metadata replay.

Technique for replicating oplog index among nodes of a cluster

Disclosed is an improved approach to implement virtualization objects in a virtualization system. The virtualization object from a first namespace is cloned as a snapshot that is accessible within a second namespace. To implement this, the virtualization object can be mounted as a target (implemented as a snapshot) that is locally accessible to the host.

Method and system for implementing virtual machine images

Systems and methods for rebalancing storage-oriented workloads across resources of a distributed data storage facility. Communications are initiated between a client computing device and a plurality of storage target devices of a distributed storage system. The distributed storage system comprises one or more computing nodes that can host virtualized controllers. A client computing device interfaced with the distributed storage system uses an IP address to access an initial virtualized controller. Upon detecting a change event, conditions of the computing environment are analyzed to determine rebalancing options. Analysis of the rebalancing options results in identification of an alternative virtualized controller. A redirect message that identifies the alternative virtualized controller is sent to the computing device. The client computing device connects to the alternative virtualized controller. Messages are sent between the client computing device and the alternative virtualized controller to access one of the storage target devices for performing storage I/O protocols.

Rebalancing storage I/O workloads by storage controller selection and redirection

Systems and methods for storage resource and computation resource expansion. A method embodiment includes migrating a computing task from an external computing environment to a different computing/storage environment. The method commences by identifying a storage system having virtualized controllers and by identifying a computing device that performs a workload that interfaces with the storage system. The virtualized controllers execute in the second computing environment to manage access to storage target devices by accessing a storage target device identified by an IP address. A particular virtualized controller that is connected to the storage target device is selected and configured to process storage I/O from a migrated workload. A user virtual machine or user executable container is configured to execute the workload on one of the nodes in the computing and storage system within the second computing environment. After migration, the computing task from the external computing environment is discarded or decommissioned.

Efficient scaling of computing resources by accessing distributed storage targets

Communication between VMs running in a clustered virtualization environment and a distributed storage system is managed. The distributed storage system consists of multiple storage/compute nodes. A hypervisor communicates with a controller in a local node on behalf of a VM. The controller will redirect a connection to a second controller on different node. The controller to which the connection is redirected is selected so as to distribute the load evenly across the cluster.

Hypervisor attached volume group load balancing

Systems and methods for managing access to storage devices in a distributed data storage environment. Embodiments operate to manage communications between a client computing device and storage target devices in a distributed storage system. The distributed storage system comprises one or more computing nodes and at least one storage target device. A client computing device interfaced with the distributed storage system uses an IP address to access a leader virtualized controller. Upon receipt of a storage access protocol message by the leader virtualized controller elected from a set of virtualized controllers, a redirect message comprising a second IP address that identifies a second virtualized controller is sent to the client computing device. The client computing device connects to the second virtualized controller. Messages are sent between the client computing device and the second virtualized controller, which in turn accesses the storage target device to carry out storage I/O protocol messaging.

Efficient scaling of distributed storage systems

A method for maintaining a mapping structure for maintaining metadata for snapshots in a virtualized storage environment, includes taking a snapshot of a virtual disk, generating an entry in a metadata structure for the snapshot, wherein the entry includes metadata for blocks in the snapshot that have been modified since a preceding snapshot and lazily generating an entry in the mapping structure for the snapshot, wherein the entry includes values for each block in the snapshot, wherein a value for a block indicates a presence of metadata in the metadata structure for the block or an absence of metadata in the metadata structure for the block.

Mapping structure for maintaining metadata for snapshots in a virtualized storage environment

Facilitating deduplication of data in a computing system without managing access to reference count variables. A method embodiment commences upon detecting first data unit and calculating a first checksum value. At a later time, a second data unit is received and the system calculates a second checksum value. If the second checksum value is the same as the first checksum value, then the first data unit and the second data unit are the same data and need not be duplicated. In such cases, an entry in the metadata points to the location of the first data unit that is already stored. Additional metadata entries are made in the metadata to associate a Boolean usage state flag and a Boolean deletion state flag with the second checksum value. Periodically scans of the metadata are performed. When both Boolean flags are in a particular state, the deduplicated data is deleted.

Growing community of inventors

Campbell, CA, United States of America

Tabrez Parvez Memon