Amazon Technologies, Inc.

Matthew Joseph Zulak

Fan Ping

Tao Chen

Shobha Agrawal

Lakshmi N Pallikila

Marc John Brooker

Divya Ashok Kumar Jain

Madhuvanesh Parthasarathy

Dylan Owen Marriner

Julian Embry Herwitz

Mauricio Roman

Guanghao Ding

Sanath Kumar Ramesh

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. The nodes of the data replication groups may generate logs containing information corresponding committed operations performed by the nodes. These logs may be collected and processed to obtain useful information corresponding to the operation of the data replication group. Furthermore, this processed information may be provided in the form of a stream to enable event driven operations corresponding to the logs.

Distributed log processing for data replication groups

Computing resource service providers operate various services used to provide functionality to customers and other computing resources operated by various entities. These services may use consistent hash rings to direct a request to a particular worker. Previous versions of the consistent hash ring may be maintained such that predecessor information can be determined. When a request is received the request and predecessor information may be provided to a worker manager. The worker manager may utilize the predecessor information to obtain state information from a predecessor worker manager. The state information may then be used to process the request.

Safe deployment using versioned hash rings

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. In order to increase the speed of provisioning, a data replication may be provision with less than the total number of nodes to be included in the group. Once an initial set of nodes have been provisioned and are implementing a consensus protocol, the data replication group may begin to receive traffic. Furthermore, new nodes may be added to the group until the total number of nodes for the group has been reached.

Dynamic provisioning for data replication groups

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. During the execution of the set of nodes various nodes may become stale or otherwise obtain a state that is inconsistent with at least one other node of the data replication group. A catch up algorithm may be employed in which a set of teachers is initialized, the various node which may be stale may select a teacher from the set of teachers and perform learning operations. This process may be repeated until the state of the various nodes is current with at least one other node of the data replication group.

Unsupervised round robin catch up algorithm

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. In order to monitor and debug the operation of the data replication group a chaining mechanism may be utilized during log creation by the set of nodes of the data replication group. The chaining mechanism may cause entries in the log to indicate an operation being performed and an operation performed immediately prior to the operation being performed. In various embodiments, an outside observer receives the logs and checks the logs for errors indicated by the chaining mechanism.

Chaining log operations in data replication groups

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. In order to increase the speed of provisioning, a pool of data replication groups may be maintained. When a request is received a data replication group may be selected from the pool, removed from the pool, and associated with the request. Furthermore, a replacement data replication group may be provisioned and added to the pool.

Growing community of inventors

Seattle, WA, United States of America

Matthew Joseph Zulak