Amazon Technologies, Inc.

Parikshit Shivajirao Pol

Akshat Vig

Rama Krishna Sandeep Pokkunuri

Rajaprabhu Thiruchi Loganathan

Subramanian Sankara Subramanian

Swaminathan Sivasubramanian

Gopinath Duddi

Safeer Mohiuddin

Ronny Weiser

Sudarshan Narasimhan

Harini Chandrasekharan

Grant Alexander Macdonald McAlister

Barry Bailey Hunter, Jr

Allan Henry Vermeulen

Timothy Andrew Rath

Kiran-Kumar Muniswamy-Reddy

Timothy Daniel Cole

Lokendra Singh Panwar

Rama Krishna Pokkunuri

In response to determining that continuous data protection is to be enabled for a particular table of a database service, a service component verifies that automated transmission of change records of the table to a log-structured journal has been configured. A given change record comprises a before-image and an after-image associated with a committed database write, and is assigned a unique sequence number. In response to a determination to restore the table as of a specified point in time, a restore record set is identified from the journal with respect to a selected snapshot of the table. The restore record set includes change records which are not represented in the snapshot and are to be represented in the restored table. A restore result table is created using the selected snapshot and the restore record set.

Scalable log-based continuous data protection for distributed databases

Information describing changes to a collection of items maintained by a database may be stored in a log file. The information in the log file may be converted into a stream of records describing the changes. The records may be directed to a computing node selected for performing a trigger function in response to the change, based on applying a hash function to a portion of the record, identifying a hash space associated with a value output by the hash function, and mapping from the hash space to the selected computing node.

Distributed stream-based database triggers

In response to determining that a secondary index is to be created for a particular table of a non-relational database service, a service component verifies that automated transmission of change records of the table to a log-structured journal has been configured. A given change record comprises a before-image and an after-image associated with a committed database write, and is assigned a unique sequence number. A change record propagator for the secondary index examines change records stored in the journal, and transmits selected change records to an index storage node. The index storage node builds the secondary index using the selected change records.

Scalable log-based secondary indexes for non-relational databases

Shared resources can be isolated such that abuse of the resource by one user does not significantly affect the use of that resource by another user. A combination of access and flow control can be used, wherein a control gateway or other such component sits along a path between the user and the resource, such that the user obtains connections or access to the resource through the gateway. In this way, the gateway can control aspects such as the number of concurrent threads or channels granted to a user, as well as any delay in providing these threads or channels to the resource. A closed feedback loop can provide real-time information such that adjustments can be made dynamically, preventing abuse by users while also preventing the resource allocations from being underutilized.

Dynamic isolation of shared resources

A first transaction manager of a partitioned storage group stores a first conditional commit record for a first write of a multi-partition transaction based on a first conflict detection operation. A second transaction manager stores a second conditional commit record for a second write of the transaction based on a second conflict detection operation. A client-side component of the storage group determines that both writes have been conditionally committed, and stores an unconditional commit record in a commit decision repository. A write applier examines the first conditional commit record and the unconditional commit record before propagating the first write to the first partition.

Scalable log-based transaction management

The monitoring and recovery of data instances, data stores, and other such components in a data environment can be performed automatically using a separate control environment. A monitoring component of the control plane can include a set of event processors for monitoring a workload of the data environment, where an event processor detecting a problem in the data plane can cause a recovery workflow to be generated in order to recover from the detected problem. The event processors can communicate with each other such that if one of the event processors becomes unavailable, the other event processors in a set are able to automatically redistribute responsibility for the workload.

Monitoring and automated recovery of data instances

Systems and methods for providing resource isolation in a shared computing environment using reinforcement learning (RL) techniques are disclosed. A resource isolation mechanism may be applied in a shared storage system, or database service, that limits the resource utilization of each namespace to its specified allocation. For example, the mechanism may be used to limit the I/O utilization of database applications in a shared computing system to a specified limit, and may apply RL techniques, along with domain-specific constraints, to the system to automatically control the rate of queries made by an application. RL techniques, such as those based on the State-Action-Reward-State-Action (SARSA) method, may be effective in controlling resource utilization for different workloads, and may be enhanced by their application in conjunction with domain-specific business rules. RL techniques may be applied globally by the service, or may be applied to particular subscribers, applications, shared resources, namespaces, or query types.

Resource isolation using reinforcement learning and domain-specific constraints

The monitoring and recovery of data instances, data stores, and other such components in a data environment can be performed automatically using a separate control environment. A monitoring component of the control plane can include a set of event processors for monitoring a workload of the data environment, where an event processor detecting a problem in the data plane can cause a recovery workflow to generated in order to recover from the detected problem. The event processors can communicate with each other such that if one of the event processors becomes unavailable, the other event processors in a set are able to automatically redistribute responsibility for the workload.

Growing community of inventors

Seattle, WA, United States of America

Parikshit Shivajirao Pol