Amazon Technologies, Inc.

Rajaprabhu Thiruchi Loganathan

Akshat Vig

Rama Krishna Sandeep Pokkunuri

Wei Xiao

Timothy Andrew Rath

William Alexander Stevenson

Srivaths Badrinath Copparam

Clarence Wing Yin Ng

Maximiliano Maccanti

Parikshit Shivajirao Pol

Subramanian Sankara Subramanian

Shyam Krishnamoorthy

Nishanth Shankaran

Clint Joseph Sbisa

Rajat Arya

Dong Shou

Gopinath Duddi

Safeer Mohiuddin

Sudarshan Narasimhan

Harini Chandrasekharan

Adam Douglas Morley

Sharatkumar Nagesh Kuppahally

Ravi Math

Lokendra Singh Panwar

Stefano Stefani

Tate Andrew Certain

James Christopher Sorenson

Vaibhav Jain

Alexander Richard Keyes

Ming-Chuan Wu

Anshul Gupta

Krishnan Seshadrinathan

Maximliano Maccanti

Rama Krishna Pokkunuri

Shyam Kirshnamoorthy

In response to determining that continuous data protection is to be enabled for a particular table of a database service, a service component verifies that automated transmission of change records of the table to a log-structured journal has been configured. A given change record comprises a before-image and an after-image associated with a committed database write, and is assigned a unique sequence number. In response to a determination to restore the table as of a specified point in time, a restore record set is identified from the journal with respect to a selected snapshot of the table. The restore record set includes change records which are not represented in the snapshot and are to be represented in the restored table. A restore result table is created using the selected snapshot and the restore record set.

Scalable log-based continuous data protection for distributed databases

Consistent queries to replicated data may be performed according to global clock values. Time values for updates performed at a data set may be identified according to a global clock. The updates may be replicated to a replicated portion of a data set. Queries directed to the replicated portion of the data set may be performed in response to determining that the time values for updates indicate that the replicated portion of the data set is consistent with a time value identified for the query according to the global clock.

Global clock values for consistent queries to replicated data

A system that implements a data storage service may store data for a database table in multiple replicated partitions on respective storage nodes. In response to a request to back up a table, the service may back up individual partitions of the table to a remote storage system independently and (in some cases) in parallel, and may update (or create) and store metadata about the table and its partitions on storage nodes of the data storage service and/or in the remote storage system. Backing up each partition may include exporting it from the database in which the table is stored, packaging and compressing the exported partition for upload, and uploading the exported, packaged, and compressed partition to the remote storage system. The remote storage system may be a key-value durable storage system in which each backed-up partition is accessible using its partition identifier as the key.

Backup of partitioned database tables

Information describing changes to a collection of items maintained by a database may be stored in a log file. The information in the log file may be converted into a stream of records describing the changes. The records may be directed to a computing node selected for performing a trigger function in response to the change, based on applying a hash function to a portion of the record, identifying a hash space associated with a value output by the hash function, and mapping from the hash space to the selected computing node.

Distributed stream-based database triggers

A system that implements a data storage service may store data for database tables in multiple replicated partitions on respective storage nodes. In response to a request to back up a table, the service may export individual partitions of the table from the database and package them to be independently uploaded (e.g., in parallel) to a remote storage system (e.g., a key-value durable storage system). Prior to uploading the exported and packaged partitions to the remote storage system, the service may verify that the exported and packaged partitions can be subsequently restored, which may include unpackaging and/or re-inflating the exported and packaged partitions to create additional unpackaged copies of the partitions, re-importing the additional unpackaged copies of the partitions into the database (e.g., as additional replicas), and/or comparing checksums generated for the exported partitions with checksums generated for the additional unpackaged copies of the partitions.

Verification of database table partitions during backup

Scalable architecture for propagating updates may be implemented for data replicated from a data set. A node may receive updates to items in a data set that have been committed to the data set. The node may determine whether the update should be applied to a replicated portion of the data set. For updates that should be applied, the node may identify another node that hosts the replicated portion of the data set and send a request to the other node to perform a conditional atomic operation to apply the update to the item in the replicated portion of the data set. The condition may compare a version identifier associated with an update and a current version identifier for the item at the other node. If the condition evaluates true, then the update to the item in the replicated portion may be performed.

Scalable architecture for propagating updates to replicated data

A system that implements a data storage service may store data for database tables in multiple replicated partitions on respective storage nodes. In response to a request to restore a given table that was backed up in a remote storage system (e.g., key-value durable storage system), the service may create a new table, and may import a copy of each of the partitions of the given table from the remote storage system into the new table. The request to restore the table may specify a modified value for a configuration parameter for the table or for one of its partitions. The service may apply the new configuration parameter value to the table or its partitions during the restore operation. The new configuration parameter value may indicate an increase or decrease in storage capacity or throughput capacity, and its application may automatically trigger a partition split or move operation.

Restoring partitioned database tables from backup

In response to determining that a secondary index is to be created for a particular table of a non-relational database service, a service component verifies that automated transmission of change records of the table to a log-structured journal has been configured. A given change record comprises a before-image and an after-image associated with a committed database write, and is assigned a unique sequence number. A change record propagator for the secondary index examines change records stored in the journal, and transmits selected change records to an index storage node. The index storage node builds the secondary index using the selected change records.

Scalable log-based secondary indexes for non-relational databases

A cache or other cluster is configuration-aware such that initialization and changes to the underlying structure of the cluster can be dynamically updated for use by a client. A client may use a client driver as an intermediary that is responsible for managing the communication with the cluster. For example, a client driver may resolve an alias from a static configuration endpoint to a storage node. The client driver may request an initial configuration from the storage node and then update configuration from one or more storage nodes that store current configuration of the cluster.

Distributed caching cluster client configuration

A management system may enable and monitor a cache or other cluster to make the cluster configuration-aware such that initialization and changes to the underlying structure of the cluster can be dynamically updated. For example, a distributed memory caching system may provide initial configuration to a client from a memory caching node referenced by an alias provided by a configuration endpoint. Updates of configuration may be retrieved from memory caching nodes, each storing current configuration of the cache cluster. A management system monitors changes to the cache cluster, such as provisioning of new caching nodes, and updates the configuration stored in the caching nodes for retrieval by a client.

Distributed caching cluster management

Growing community of inventors

Issaquah, WA, United States of America

Rajaprabhu Thiruchi Loganathan