Amazon Technologies, Inc.

Carlos Vara Callau

Allan Henry Vermeulen

Christopher Richard Jacques De Kadt

Timothy Daniel Cole

Benjamin Warren Mercier

Sayantan Chakravorty

Aaron Gifford Freshwater

Alan Michael Judge

Tyson James Lamoreaux

Karl Andre McCabe

Timothy Richard Kohn

Sonali Roy

Eoin Francis Cavanagh

Rohit Kulshreshtha

Live migration of log-based consistency mechanisms may be implemented for data stores. A log describing changes to replicas of data maintained at different nodes may be implemented in different ways. While the nodes are available for servicing access requests, a migration of consistent maintained between one instance of a log and a different instance of the log may be performed. A change may be committed to the current instance of the log identifying the new instance of the log as describing changes to the log subsequent to the migration. Upon completing migration, the data may be consistently maintained according to the different instance of the log.

Live migration of log-based consistency mechanisms for data stores

A journal service maintains a journal that holds a number of journal entries. Access to the journal entries is controlled by providing a lead resource with access to a named role associated with the journal. A pool of alternate resources containing potential replacements for the lead resource is maintained. A succession order may be established for the pool of alternate resources, and individual resources in the pool of alternate resources may be managed based at least in part on their position within the succession order. The succession order may be established using various criteria that are indicative of a likelihood that a particular alternate resource will be available if the lead resource fails, such as the presence of shared points of failure between the current lead resource and an alternate resource. The lead resource may be replaced with an alternate resource as a result of various impeachment events.

Management of alternative resources

A journal service manages a journal that holds a number of journal entries. Journal entries are provided to the journal service by one or more producer services, and the journal service provides journal entries to one or more consumer services. Lead consumers retrieve journal entries from the journal service by specifying a named cursor which identifies a location within the journal from which the journal entries are retrieved. The journal service may select a lead producer and a lead consumer from a collection of candidate producers and consumers using a fitness score determined for each candidate producer and consumer. The fitness score may be based at least in part on the processing capability, predicted reliability, and cost of each candidate producer and consumer. The journal service may replace a lead consumer or producer with an alternate consumer or producer in response to a variety of events.

Journal service with named clients

Technologies are disclosed for improving the availability of a distributed application by computing scores for the application that indicate the extent to which the configuration of hosts implementing the application complies with best practices for maximizing availability. One score can be generated for the application based upon the physical location of the hosts used to implement the application and the components upon which the application is dependent. Another score can be generated for the application based upon the number of data centers that hosts executing the application are located in as compared to a minimum required number of data centers. Another score can be generated for the application based upon the distribution of hosts utilized to execute the application across data centers. Operations can be initiated for improving the scores, such as migrating a host to a new rack or server, or migrating a host to a different data center.

Availability of a distributed application using diversity scores

A resource manager arranges the resources in a computer system into one or more resource pools. The resource manager allocates a number of active resources and a number of backup resources to a particular resource pool. For each resource managed by the resource manager, the resource manager acquires information that describes the capacity and reliability of the resource. Capacity and reliability information for the particular resource pool is determined based on the capacity and reliability information associated with the resources assigned to the pool. In response to a request, the resource manager may provide an application with resources from several resource pools. The likelihood that the resource manager will be able to provide sufficient resources to the application may be determined based at least in part on the reliability information associated with the several resource pools.

Resource manager

Generally described, systems and methods are provided for detecting the impact of network failures. The system collects performance information from a plurality of nodes and links in a network, aggregates the collected performance information across paths in the network, processes the aggregated performance information for detecting failures on the paths, adjusts the set of performance information by removing the performance information for any nodes considered to be associated with performance information that is statistically different from performance information from other nodes at a given location or extrapolates the collected information to other paths, and determines the impact to customers of the network failures detected using the adjusted set of performance information.

Determining impact of network failures

A pipeline management service improves the operation of a processing pipeline by monitoring the operation of one or more journal services within the processing pipeline, and adjusting operational parameters of the one or more journal services. In various examples, the pipeline management service determines a model of the processing pipeline by identifying the producers and consumers associated with each of the one or more journal services, and determines a processing flow through the one or more journal services based at least in part on the identities of the associated producers and consumers. The pipeline management service acquires backlog information from each of the one or more journal services, and may adjust the operational parameters of a journal service associated with an identified processing bottleneck. In some examples, the pipeline management service adjusts capacity and reliability parameters of the one or more journal services to improve the operation of the processing pipeline.

Management of a data processing pipeline

A distributed system uses a consensus protocol to maintain a data set shared by nodes of the distributed system. When a change is proposed to the data set, the change must pass multiple tests, including a test regarding the validity of the proposed change according to the consensus protocol and a test regarding the validity of the proposed change according to an application that utilizes the data set. As a result, poison pills that could harm operation of the distributed system are prevented from being accepted, regardless of whether such poison pills would satisfy any conditions of the consensus protocol.

Poison pill prevention for consensus protocols

A jury candidate server (JCS) of a distributed state manager (DSM) implements a plurality of jury configuration evaluation iterations. In a particular iteration, the JCS generates one or more jury configuration options (JCOs) representing changes to a current jury, and assigns quality metrics to the JCOs based at least in part on a prioritized set of criteria among which location diversity has the highest priority. If a JCO meets a quality threshold, a corresponding jury transition is proposed for approval by the current jury.

Distributed state manager jury selection

A system and methods for monitoring and determining agent and network health, having a network monitoring and failure detection system that collects data reports and accumulates a set of data defined in terms of a time window. The network monitoring and failure detection system makes a determination of the agent health during the time window. The network monitoring and failure detection system then processes the collected data based on the determined health. The processing of the collected data can include disregarding the data, weighing the data, filtering the data, using the data in a feedback loop, or processing the data using another method or algorithm.

Growing community of inventors

Seattle, WA, United States of America

Carlos Vara Callau