Amazon Technologies, Inc.

Oracle International Corporation

Deepak Agarwal

Anurag Windlass Gupta

Jakub Kulesza

Tushar Jain

Don Johnson

Srividhya Srinivasan

Stefano Stefani

Aleksandras Surna

Kent Lee

Qiang Wang

Travis Portz

Zelaine Fong

Bharath Kumar Chelepalli

Haining Mo

Zelanie Fong

The present embodiments relate to throttling input/output (IO) processing tasks at a centralized node (e.g., a SmartNIC device) and implement multiple routing paths to prevent performance degradation in handling data patterns. For instance, the centralized node can implement a throttling system to assign portions of a number of input/output operations to be performed to a series of extent servers in a cloud infrastructure environment. The present embodiments also can implement multiple paths and/or multiple path groups to route the assigned portions of the I/O operations. For instance, the assigned portions of I/O operations can be routed to the extent servers via a first path of multiple paths. In the event of a failure of the first path, the assigned portions of I/O operations can be re-directed to a second path or another path in the path group.

Increased data processing performance of a non-volatile memory express (NVME) block store

A distributed data warehouse system maintains data blocks on behalf of clients, and stores primary and secondary copies of data blocks on different disks or nodes in a cluster. The data warehouse system may back up data blocks in a key-value backup storage system. In response to a query targeting a data block previously stored in the cluster, the data warehouse system may determine whether a consistent, uncorrupted copy of the data block is available in the cluster (e.g., by applying a consistency check). If not (e.g., if a disk or node failed), the data warehouse system may automatically initiate an operation to restore the data block from the backup storage system, using a unique identifier of the data block to access a backup copy. The target data may be returned in a query response prior to restoring primary and secondary copies of the data block in the cluster.

Automatic repair of corrupted blocks in a database

A stateful cluster may implement scaling of the stateful cluster while maintaining access to the state of the stateful cluster. A scaling event for a stateful cluster may be detected, and in response the stateful cluster may be adjusted to include a different number of nodes. The state of the cluster may then be logically distributed among the different number of nodes according to a monotone distribution scheme. The adjusted node may then service access requests according to the monotone distribution scheme. Prior to making the adjusted storage cluster available for servicing access requests, the nodes from the original cluster may still service access requests for state.

Scaling stateful clusters while maintaining access

A distributed database system may perform a single phase commit for transactions involving updates to multiple databases of the distributed database system. A client request may be received that involves updates to multiple database of the distributed database system. The updates may be performed at a front-end database and a back-end database. Log records indicating the updates to the front-end database may be sent to the back-end database. The log records and the updates performed at the back-end database may be committed together as a single phase commit at the back-end database. In the event of a system failure of the front-end database, log records may be requested and received from the back-end database. A restoration of the front-end database may be performed based, at least in part, on the received log records.

Single phase transaction commits for distributed database transactions

A data store may implement incremental out-of-place updates to a dataset. A dataset may maintain data across different storage locations linked together according to an ordering schema for servicing queries. As updates to the dataset are received, the updates may be persisted but not maintained in-place. In order to update the data store and maintain the ordering schema, incremental updates to the dataset may be performed without blocking queries directed toward the dataset. The dataset may be divided into multiple data chunks that correspond to different storage locations and an updated version of the data chunk may be generated in new storage locations. The new storage locations may then replace the storage locations of the prior version of the data chunk in order to link the new storage locations to the other linked storage locations in the dataset for servicing queries.

Incremental out-of-place updates for datasets in data stores

A distributed data warehouse system may maintain data blocks on behalf of clients in multiple clusters in a data store. Each cluster may include a single leader node and multiple compute nodes, each including multiple disks storing data. The warehouse system may store primary and secondary copies of each data block on different disks or nodes in a cluster. Each node may include a data structure that maintains metadata about each data block stored on the node, including its unique identifier. The warehouse system may back up data blocks in a remote key-value backup storage system with high durability. A streaming restore operation may be used to retrieve data blocks from backup storage using their unique identifiers as keys. The warehouse system may service incoming queries (and may satisfy some queries by retrieving data from backup storage on an as-needed basis) prior to completion of the restore operation.

Growing community of inventors

Redmond, WA, United States of America

Deepak Agarwal