Vmware, Inc.

Nicira, Inc.

Maithem Munshed

Medhavi Dhawan

Michael Wei

Dahlia Malkhi

Anny Martinez Manzanilla

Zeeshan Altaf Lokhandwala

Srinivas Neginhal

Amar Padmanabhan

Vadim Spivak

Cheng Zhang

Ragnar Edholm

Michi Mutsuzaki

Roger Michoud

Rajneesh Bajpai

Ming Wen

Vjekoslav Brajkovic

Jiaqi Chen

Saeed A Behnam

David Tsai

Xin Liang Li

Wenbin Zhu

Luca Castellano

Xiaoqin Ma

Vui Chiap Lam

Anfernee Yongkun Gui

Badhri Sampathkumar

Sundar Sridharan

Pavan Kumar Anumula

Certain embodiments described herein are directed to methods and systems for adding one or more nodes to a first cluster including a first node in a computer system. A method performed by the first node comprises receiving a first request from a second node to join the first cluster. The method also comprises retrieving a first cluster configuration associated with the first cluster from a distributed database through a first database server (DBS) and creating a second cluster configuration using the first cluster configuration and information received from the second node as part of the request. The method further comprises populating a first one or more local trust stores of a first one or more processes executing on the first node with a second one or more security certificates of a second one or more processes executing on the second node. The method further comprises writing the second cluster configuration to the distributed database and returning the second cluster configuration to the second node.

Methods and systems for securely and efficiently clustering distributed processes using a consistent database

Garbage collection processing in a distributed shared log system includes a client identifying obsoleted log entries for a shared data object. The client sends information associated with the identified obsoleted log entries to a shared log server. The shared log server receives information associated with obsoleted log entries from all clients in the distributed shared log system and uses the information to delete the obsoleted log entries. The shared log server can update a snapshot mark to indicate the earliest time that a snapshot of the shared log can be taken. The snapshot mark can be updated based on the information associated with obsoleted log entries.

In-place garbage collection for state machine replication

A shared log system can provide a key-index structure (main map) that allows clients (users) to access the state of objects stored in the shared log system. The client can define secondary indices on the main map to provide access to the data that are tailored to the client's data needs. Each client can generate their own customized set of secondary indices different from other clients.

Just-in-time multi-indexed tables in a shared log

Some embodiments provide a method that, at a first machine that operates at a first physical site as a primary global manager for a logical network spanning a plurality of physical sites, receives a set of ordered data describing a configuration of the logical network. The method stores, in a particular order, the received set of ordered data in a first database located at the first physical site. Through a channel between (i) the first machine and (ii) a second machine that operates at a second physical site as a secondary global manager for the logical network in case of failure of the primary global manager, the method provides the set of ordered data in the particular order for the second machine to store in a second database in the particular order. The second database located at the second physical site.

Replication of logical network data between global managers

Disclosed are a method and system for implementing a distributed persistent queue. To implement the queue, an API is defined that includes a command to enqueue an object onto the queue, to obtain the ordering of all objects in the queue and to remove any item from the queue. Enqueuing an object includes obtaining a unique ID for the object from a generator and then storing the object in a persistent shared log, which provides persistent storage for the queue. Obtaining an ordering of all objects in the persistent queue includes receiving for each object stored in the queue a pair of values, the first being the unique ID and the second being a sequence number from the persistent shared log. Regardless of the host computer system making the request for the ordering of objects in the queue the ordering provided to each host computer system is the same.

Distributed persistent queue facilitating state machine replication

Techniques disclosed herein provide an approach for managing aggregation service hierarchies. In some embodiments, a hierarchy of an aggregation service is identified. The hierarchy comprises a plurality of nodes, where a respective node is associated with at least one host computer. The aggregation service places resource consumers based on the nodes. A host computer is assigned as a child host of a leaf node based on a clustering heuristic. The clustering heuristic requires the host computer to have access to at least one resource that is accessible to an existing child host of the leaf node. A resource consumer associated with the leaf node is executed on the host computer.

Creating and managing aggregation service hierarchies

A log unit provides a shared log for recording updates on data objects. Garbage collection is performed locally and in-place by the log unit. In a marking portion of the garbage collection process, the log unit identifies and marks log entries that record supersedable updates. In a deallocation portion of the process, the log unit deallocates at least portions of the marked log entries that contain supersedable updates.

In-place garbage collection of a sharded, replicated distributed state machine based on supersedable operations

Certain embodiments described relate to methods and systems for adding one or more nodes to a first cluster including a first node in a computer system. A method performed by the first node comprises retrieving a first cluster configuration associated with the first cluster from a distributed database through a first database server (DBS) and creating a second cluster configuration using the first cluster configuration and information received from a second node as part of a request to join the first cluster. The method further comprises populating a first one or more local trust stores of a first one or more processes executing on the first node with a second one or more security certificates of a second one or more processes executing on the second node. The method further comprises writing the second cluster configuration to the distributed database and returning the second cluster configuration to the second node.

A log unit provides a shared log for recording updates on data objects. Garbage collection is performed locally and in-place by the log unit. In a marking portion of the garbage collection process, the log unit identifies and marks log entries that record mergeable updates. In a deallocation portion of the process, the log unit merges one or more mergeable log entries and deallocates at least portions of the merged log entries.

In-place garbage collection of a sharded, replicated distributed state machine based on mergeable operations

Techniques for implementing dynamic timeout-based fault detection in a distributed system are provided. In one set of embodiments, a node of the distributed system can set a timeout interval to a minimum value and transmit poll messages to other nodes in the distributed system. The node can further wait for acknowledgement messages from all of the other nodes, where the acknowledgement messages are responsive to the poll messages, and can check whether it has received the acknowledgement messages from all of the other nodes within the timeout interval. If the node has failed to receive an acknowledgement message from at least one of the other nodes within the timeout interval and if the timeout interval is less than a maximum value, the node can increment the timeout interval by a delta value and can repeat the setting, the transmitting, the waiting, and the checking steps.

Dynamic timeout-based fault detection

A shared log in a distributed system provides for direct access to the most current data state of an object stored in the shared log. Directly accessing the data state of an object obviates the need for a client to replay all the transactions stored in the shared log made on the object.

Direct access to object state in a shared log

In accordance with disclosed embodiments, a shared log system includes a sequencer to verify transactions that comprise a source object and one or more members of the source object (source data objects), a target object and one or more members of the target object (target data objects), and a snapshot time reference. The sequencer verifies transaction using the snapshot time to determine whether the data states of the source data objects at the time of the snapshot time represent their most current data state in a shared log.

Fine-grained conflict resolution in a shared log

Techniques disclosed herein provide an approach for assigning resource consumers to available resources. In one embodiment, components of a distributed scheduler are organized into a hierarchy, such as a tree. A placement request received at a root scheduler of the hierarchy is propagated down the hierarchy, either to all children or to randomly selected subsets of children of each scheduler in the hierarchy. Leaf schedulers in the hierarchy that receive the request each propagate back up a score indicating the amount of free resources in its corresponding resource bucket. Branch schedulers then compare scores that they receive, and each further propagate one of the received scores, such as the highest score, based on the comparison, until the root scheduler is reached. The root scheduler makes an additional comparison and returns one of the resource buckets in response to the received placement request.

Distributed hierarchical scheduler

Embodiments of the present disclosure relate to techniques for maintaining a state of a distributed system. In particular, certain embodiments relate to identifying a function. Some embodiments relate to, upon determining that the function comprises an annotation indicating that the function is capable of modifying the state of the distributed system, transforming the function to allow the function to generate updates to a state machine.

Annotation-driven framework for generating state machine updates

Techniques disclosed herein provide an approach for creating and managing aggregation service hierarchies, such as hierarchies used in distributed scheduling services and heartbeat services. In one embodiment, management nodes accept host computer registration events and add host computers to a hierarchy used as the aggregation mechanism in an aggregation service. The management nodes each manage a portion of the hierarchy and configure registered hosts to take the roles of leaf, branch, and root nodes in the hierarchy. Further, the management nodes dynamically mutate the hierarchy by reassigning host roles, in response to host additions and failures, thereby maximizing fault tolerance/high availability and efficiency.

Growing community of inventors

Palo Alto, CA, United States of America

Maithem Munshed