Amazon Technologies, Inc.

Abhishek Rajnikant Sinha

Luca Natali

Bhargava Ram Kalathuru

Rahul Sharma Pathak

Xing Wu

Jason Douglas Denton

Jian Fang

Turkay Mert Hocanin

Pratik Bhagwat Gawande

Yuanyuan Yue

Sumeetkumar Veniklal Maru

Armen Tangamyan

Yufeng Jiang

Saurabh Dileep Baji

Jonathan Daly Einkauf

Andrew Edward Caldwell

Marc Howard Beitchman

George Steven McPherson

Ankit Kamboj

Dag Stockstad

A service provider may apply customer-selected or customer-defined auto-scaling policies to a cluster of resources (e.g., virtualized computing resource instances or storage resource instances in a MapReduce cluster). Different policies may be applied to different subsets of cluster resources (e.g., different instance groups containing nodes of different types or having different roles). Each policy may define an expression to be evaluated during execution of a distributed application, a scaling action to take if the expression evaluates true, and an amount by which capacity should be increased or decreased. The expression may be dependent on metrics emitted by the application, cluster, or resource instances by default, metrics defined by the client and emitted by the application, or metrics created through aggregation. Metric collection, aggregation and rules evaluation may be performed by a separate service or by cluster components. An API may support auto-scaling policy definition.

Automatic scaling of resource instance groups within compute clusters

Interruptible computing resources for executing queries may be selected. When queries are received, the probability that query can complete execution at interruptible computing resources may be determined. Computing resources to process the query may be selected based on the probability. For queries that are performed on computing resources that are interrupted, interruption handling techniques may retry the query on other computing resources or complete the query on other computing resources, in some embodiments.

Selecting interruptible resources for query execution

Queries may be received and executed by a managed query service. A query directed to data sets that are separately stored in a remote data store may be received. Computing resources to execute the query may be provisioned from a pool of computing resources that are configured to execute queries. The query may be routed to the provisioned computing resources to execute the query. Results may be obtained from the computing resource and provided to a submitter of the query.

Managed query service

Query execution status may be tracked to selectively route queries to resources for execution. The completion of queries executing at computing resources obtained from a pool of computing resources configured to execute queries may be detected. Instead of returning the computing resources to the pool, the computing resources may be identified as available in resource management data. When another query is received, the resource management data may be evaluated to select an available computing resource. The query may then be routed to the selected computing resource for execution.

Tracking query execution status for selectively routing queries

The configuration of computing resources for executing queries may be selected. A comparison of the configuration of computing resources that executed previous queries may be made to select the configuration of computing resources for a received query. A historical query execution model maybe applied, in some embodiments, to determine a resource configuration for computing resources to execute a query. The computing resources may be selected from available computing resources according to the determined resource configuration.

Selecting resource configurations for query execution

A system adapted to process long-running processes is disclosed. A request to upload data is received at a server. The server divides the data into multiple parts and launches a separate process to upload each of the divided parts. The server records for each process the processing time or duration that the particular process used to upload its corresponding data item. The server maintains an average processing duration that is calculated from the processing durations of the completed processes. The server identifies that one process is continuing to run and compares a processing duration for the particular process to a threshold derived from the average processing duration. If the processing duration for the particular process exceeds the threshold, the server initiates a new process to upload the same data item. When one of either the new process or the still running process has completed processing, the server terminates the other process.

Processing of long running processes

Growing community of inventors

Redmond, WA, United States of America

Abhishek Rajnikant Sinha