Amazon Technologies, Inc.

Brian Gregory Long

Sumit Lohia

Matthew Dean Rehder

Paul A Stancik

Michal Zygmunt Zawirski

Justin Thomas Pfifer

Sunjae Chong

Robert Kennedy

Dzianis Lisouski

Karl Andre Mccabe

A computing resource monitoring service receives a command to read a metric pertaining to computing resources provided to the customer. The command may comprise a hash key for the metric. Based on the key, the service may select one or more data zones, each of the data zones comprising one or more storage nodes wherein metrics are stored. The service may be configured to obtain a schedule of storage nodes from the data zones in order to determine the active storage nodes based at least in part on the key. The service may receive a response comprising the metric from one or more storage nodes from the one or more active nodes based at least in part on the key. The service may apply agreement criteria to the responses. If the responses do not meet the criteria, the service may perform a reconciliation process to generate a compiled response.

Data reconciliation in a distributed data storage network

A computing resource monitoring service detects an issue with processing of metrics data. The metrics data is stored in observation journals within an observation journal logical data container. A computing resource service provider may configure a map reduce processor to select a prior version of an index file, the index file comprising a mapping of the metrics data stored within a metric store and a manifest comprising a listing of all processed observation journals. The map reduce processor may identify newly unprocessed observation journals and process these journals in order to generate new data objects and a new version of an index. Subsequently, an index pointer may be modified such that obsolete versions of the index are removed and the map reduce processor refers to a different version of the index file in order to support future roll back of metrics data.

Roll back of scaled-out data

A computing resource monitoring service receives metrics data from customer applications and resources provided by a computing resource service provider. The metrics data is stored in observation journals within an observation journal logical data container. A map reduce processor is configured to obtain a current index file, the index file comprising a mapping of the metrics data stored within a metric store and a manifest comprising a listing of all processed observation journals. The map reduce processor may process the unprocessed observation journals to separate, sort and aggregate metrics data stored therein. This process generates index artifacts that may be merged with the index file, in an append-only process, to generate a new index file. An index pointer may be modified such that the map reduce processor refers to a different version of the index file in order to support roll back of metrics data.

Scale-out of data that supports roll back

A computing resource monitoring service receives an executable command to redundantly store a metric pertaining to computing resources provided to the customer by a computing resource service provider. The executable command may comprise a hash key and a hash value for the metric. Based on the hash key included in the executable command, the computing resource monitoring service may select one or more data zones, each of the data zones comprising one or more storage nodes for storing metrics. The computing resource monitoring service may be configured to obtain a schedule of storage nodes from the data zones in order to determine the active storage nodes based at least in part on the hash key. Accordingly, the computing resource monitoring service may identify a storage node from the one or more active nodes based at least in part on the hash key and store the metric in the identified node.

Growing community of inventors

Seattle, WA, United States of America

Brian Gregory Long