Amazon Technologies, Inc.

Lakshmi N Pallikila

Tao Chen

Divya Ashok Kumar Jain

Fan Ping

Shobha Agrawal

Marc John Brooker

Matthew Joseph Zulak

Danny Wei

Yan Valerie Leshinsky

Kerry Quintin Lee

Madhuvanesh Parthasarathy

James Andrew Trenton Lipscomb

Tarun Goyal

Norbert Paul Kusters

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. The nodes of the data replication groups may generate logs containing information corresponding committed operations performed by the nodes. These logs may be collected and processed to obtain useful information corresponding to the operation of the data replication group. Furthermore, this processed information may be provided in the form of a stream to enable event driven operations corresponding to the logs.

Distributed log processing for data replication groups

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. The nodes may be executed by one or more host computer systems. In order to avoid failure of the data replication groups as safe deployment strategy is provided. The one or more host computer systems may be assigned to deployment groups where the number of hosts assigned to the deployment groups host less than a quorum of the nodes of a particular data replication group.

Deployment strategy for maintaining integrity of replication groups

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. In order to increase the speed of provisioning, a data replication may be provision with less than the total number of nodes to be included in the group. Once an initial set of nodes have been provisioned and are implementing a consensus protocol, the data replication group may begin to receive traffic. Furthermore, new nodes may be added to the group until the total number of nodes for the group has been reached.

Dynamic provisioning for data replication groups

Data replication groups may be used to store data in a distributed computing environment. A data replication group may include a set of nodes executing a consensus protocol to maintain data durably. In order to increase efficiency and performance of the data replication, a particular node of the data replication group may be assigned the role of master node. The role of master node may be lease in accordance with a consensus protocol. If the lease is not renewed within an interval of time election/selection of a new master node may be commenced.

Leadership lease protocol for data replication groups

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. During the execution of the set of nodes various nodes may become stale or otherwise obtain a state that is inconsistent with at least one other node of the data replication group. A catch up algorithm may be employed in which a set of teachers is initialized, the various node which may be stale may select a teacher from the set of teachers and perform learning operations. This process may be repeated until the state of the various nodes is current with at least one other node of the data replication group.

Unsupervised round robin catch up algorithm

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. In order to monitor and debug the operation of the data replication group a chaining mechanism may be utilized during log creation by the set of nodes of the data replication group. The chaining mechanism may cause entries in the log to indicate an operation being performed and an operation performed immediately prior to the operation being performed. In various embodiments, an outside observer receives the logs and checks the logs for errors indicated by the chaining mechanism.

Chaining log operations in data replication groups

Data replication groups may be used to store data in a distributed computing environment. The data replication groups may include a set of nodes executing a consensus protocol to maintain data durably. In order to increase the speed of provisioning, a pool of data replication groups may be maintained. When a request is received a data replication group may be selected from the pool, removed from the pool, and associated with the request. Furthermore, a replacement data replication group may be provisioned and added to the pool.

Replication group pools for fast provisioning

A computing system recovers volumes in a distributed computing environment while reducing downtime of storage servers. In an embodiment, a storage server contacts a control plane after a storage failure has occurred. If the storage server hosts an authoritative copy of an offline volume, the storage server is requested to restore the volume. Non-authoritative volumes are removed from the storage server and the storage server provides read access to the restored volume while resuming storage services.

Growing community of inventors

Seattle, WA, United States of America

Lakshmi N Pallikila