Amazon Technologies, Inc.

Matthew James Eddey

Anton André Eicher

Richard Alan Hamman

John Sandeep Yuhan

Mahmood Miah

Carl Bellingan

Marcin Piotr Kowalski

David R Richardson

Peter Nicholas Desantis

James Alfred Gordon Greenfield

Abhishek Kumar

Kerry Quintin Lee

Atle Normann Jorgensen

André Mostert

Jacobus Johannes Nicolaas Van Der Merwe

George Shipilov

Christopher Richard Jacques De Kadt

A fleet of databases may comprise subsets of databases sharing a common schema for replication, horizontal partitioning and other applications. Events related to potential deviations from a desired schema may be monitored. Automated means may be employed to detect variations from a desired schema. An intermediate representation of database schema may be formed and used to compare database schemas. Subclasses and allowed versions of desired schemas may be defined. Security attributes related to schema inspection and modification may be mapped to enable selected operations on schema.

Database fleet schema maintenance

A snapshot analysis system analyzes a plurality of data snapshots taken in connection with data stored on a block device allocated by a data storage system. The snapshot analysis system may include an ingestor capable of initially detecting new snapshots and adding a root node for the snapshots. The system may include a block device analyzer that analyzes each snapshot to determine its contents, the relationship within data structures extant within the snapshot, and the snapshot's relationship to other snapshots and/or that of other block devices. The system may also include a clustering analyzer capable of determining whether snapshots are associated with multipart block devices, such as LVM or MD RAID devices. The system may further include a block device emulator that exposes data associated with a given snapshot as an addressable block device without necessitating retrieval or exposure of the full block device to which the snapshot is associated.

Snapshot clustering techniques for multipart volumes

Methods and apparatus for profile-guided preloading for virtualized resources are described. A block-level storage volume whose contents are to be populated via data transfers from a repository service is programmatically attached to a compute instance. An indication of data transfers from the repository to a block storage service implementing the volume is obtained, corresponding to a particular phase of program execution at the compute instance. A storage profile is generated, based at least in part on the indication of data transfers. The storage profile is subsequently used to pre-load data from the repository service on behalf of other compute instances.

Profile-guided data preloading for virtualized resources

A system includes a storage volume configured to store a data set in a plurality of data blocks, a data store configured to store a plurality of captures of the data set in a plurality of data chunks, and file retrieval logic. The data set includes a file stored in a first data block of the plurality of data blocks. The file retrieval logic is configured to identify a first data chunk of the plurality of data chunks in which the first data block as captured in a first capture is stored in the data store, retrieve the first data chunk from the data store, and read the first data block as captured in the first capture from the first data chunk.

File extraction out of a backup

A system and method for maintaining data integrity during data transformation operations. The system and method include obtaining a message from a set of queues, obtaining, from a first data store, a data object indicated by the message, and generating, at a first host, a set of error detection codes corresponding to a transformation of the data object according to a transformation scheme. The system and method further include, transforming, at a second host different from the first host, the data object according to the transformation scheme into the transformation of the data object, verifying the transformation against the set of error detection codes, and storing the transformation in a second data store.

Ensuring data integrity during large-scale data migration

A system includes a storage volume configured to store a data set in a plurality of data blocks, a data store configured to store a plurality of captures of the data set in a plurality of data chunks, and file retrieval logic. The data set includes a file stored in a data block of the plurality of data blocks. The plurality of captures includes the file captured at different points in time. The file retrieval logic is configured to identify the plurality of data chunks in which the data block as captured in the plurality of captures is stored in the data store, retrieve the plurality of data chunks from the data store, and read the data block as captured in the plurality of captures from the plurality of data chunks to produce a plurality of file versions.

Version history of files inside a backup

Systems and methods for performing diagnostic tests within a multi-tenant environment are described. Diagnostics are performed on one or more components, such as host computing devices. The one or more components send resulting diagnostic information to an ingester which collects the diagnostic information. The ingester then sends the diagnostic information to a database. A generator is able to query the information in the database and create metrics. Based at least in part on metrics, an alarm may be triggered.

Testing framework for host computing devices

Functionality is disclosed for throughput and latency optimized volume initialization. Data blocks used for boot processes of multiple virtual machine (VM) instances are identified, for example, where the multiple VM instances are created from a specific VM image. A load order document is generated that identifies the data blocks, and also identifies an order in which the data blocks are used for the boot process. The identified data blocks are coalesced to generated a coalesced boot volume. Subsequently, each time a VM instance is created from the specific VM image, the coalesced boot volume (e.g., instead of the individual data blocks) is transmitted from a data store to the a block storage service in the order specified by the load order document. The block storage service exposes a storage volume to the VM instance that can be used to initialize the VM instance.

Throughput and latency optimized volume initialization

A snapshot analysis system analyzes a plurality of data snapshots taken in connection with data stored on a block device allocated by a data storage system. The snapshot analysis system may include an ingestor capable of initially detecting new snapshots and adding a root node for the snapshots. The system may include a block device analyzer that analyzes each snapshot to determine its contents, the relationship within data structures extant within the snapshot, and the snapshot's relationship to other snapshots and/or that of other block devices. The system may also include a clustering analyzer capable of determining whether snapshots are associated with multipart block devices, such as LVM or MD-RAID devices. The system may further include a block device emulator that exposes data associated with a given snapshot as an addressable block device without necessitating retrieval or exposure of the full block device to which the snapshot is associated.

Block device emulation for data snapshots

A technology is described for determining a launch workflow for launching multiple computing instances on a physical host in a computing service environment using a predicted launch time. An example method may include receiving a launch plan to launch a plurality of computing instances on a physical host within a computing service environment. A first launch workflow and a second launch workflow may then be determined for the launch plan where the first launch workflow and the second launch workflow may specify different sequences of operations performed to launch the computing instance on the physical host. Predicted launch times for the first launch workflow and the second launch workflow may be obtained and the predicted launch times may be compared to determine a launch workflow used in executing the launch plan.

Computing instance launch workflow

A technology is described for managing computing capacity within a computing service environment. An example method may include determining a state of a computing service environment having a group of physical hosts that execute computing instances and determining a predicted demand for computing instance launches on the group of physical hosts. A co-location of the computing instance launches on the group of physical hosts may then be determined by applying the predicted demand for computing instance launches to the state of the computing service environment. A predicted launch time for the computing instance launches may then be determined using the co-location of the computing instance launches as an input feature. An additional physical host may then be added to the group of physical hosts to decrease the predicted launch time.

Computing service capacity management

A technology is described for predicting a launch time for a computing instance. An example method may include receiving a request for a predicted launch time to launch a computing instance on a physical host within a computing service environment. Data associated with launch features of a computing instance may then be obtained, where the launch features may be determined to have an impact on a launch time of the computing instance on a physical host within a computing service environment. The launch features of the computing instance may then be input to a machine learning model that outputs the predicted launch time for launching the computing instance within the computing service environment.

Growing community of inventors

Seattle, WA, United States of America

Matthew James Eddey