Amazon Technologies, Inc.

Microsoft Technology Licensing, LLC

Tengiz Kharatishvili

Alexandre Olegovich Verbitski

Kamal Kant Gupta

Anurag Windlass Gupta

Stefano Stefani

Saileshwar Krishnamurthy

Niket Goel

Xiaofeng Bao

Alok Nandan Nikhil

Bohan Liu

Gopi K Attaluri

Pradeep Jnana Madhavarapu

Justin Jon Levandoski

Debanjan Saha

Mushahid Alam

Wei Xiao

Sandor Loren Maurice

Alexey Kuznetsov

Lawrence Douglas Webley

Hyungsoo Jung

Dhruva Ranjan Chakrabarti

Murali Brahmadesam

Atrayee Mullick

Joseph Andrew Tucek

Punit Rajgaria

Yingjie He

Shriram Sridharan

Andrew Scott Richardson

Yuhui Yuan

Aakash Shah

Yoni Shalom

Jagdeep Singh Sidhu

Siyuan Xing

Yibo Zhu

Saraswathy Narayan

Nicholas Piers O'Shea

Damien Katz

Hongbo Song

Samuel James McKelvie

Yan Valerie Leshinsky

Yi Ding

Yuri L Volobuev

Saleem Mohideen

Changan Han

Jeffrey Davis

Sailesh Krishnamurthy

Xiang Peng

Tiffany Jianto

Zhaohui Zhang

Madhura Madhukar Parikh

Volodymyr Verovkin

Dhruva Ranjan Chakrabarit

Aleksandr Valerevich Feinberg

Alexandre Olegovich Vebitski

Namrata Bapat

Kamal Kunt Gupta

Huawei Wang

Jianguo Wang

Proxy-based scaling may be performed for databases. A proxy may be implemented for a database that can establish a connection between the proxy and a database engine to perform a database queries received from a client at the proxy. A scaling event may be detected for the database responsive to which the proxy may establish a connection with a new database engine which may, in some embodiments, have different capabilities or resources that address the features or criteria that triggered the scaling event. Session state may be copied from the database engine to the new database engine so that the new database engine may be able to provide access to the database on behalf of requests received from the client through the proxy.

Proxy-based database scaling

Transaction statements may be forwarded for a database transaction. A transaction begun at a read-only node may include a write statement. The read-only node may forward the write statement to the read-only node to perform as a transaction. The read-only node may update a read view for the transaction to include the write statement before performing a read statement submitted after the write statement at the read-only node.

Selective forwarding for multi-statement database transactions

Optimistic concurrency may be implemented for multiple writers to a database. Writes to a database from different writers may succeed unless a conflict is detected at the data store for the database. For detected conflicts, a request to resolve the conflict between writes may be submitted to a primary writer, which may determine and provide a resolution of the conflict to the conflicting writers. The data store may then be updated to commit the selected write identified by the resolution of the conflict.

Optimistic concurrency for a multi-writer database

Systems and methods are disclosed to implement, in a multi-writer database system, a database engine node that optimistically sends queued purge operations to a backend storage system without waiting for acknowledgement of previous operations from the storage system. If a conflict is later detected for a particular operation, the database engine node may retry some portion of the purge operations in the queue. In embodiments, the purge operations are performed as part of a garbage collection phase of a multi-version concurrency control (MVCC) protocol, where unneeded versions of objects previously generated for isolated snapshots are deleted. In embodiments, the operations in the queue may be grouped so that a rejection of one operation in the group will cause the entire group to be retried. Because these purge operations are idempotent and directed to objects that are invisible, they may be freely retried in arbitrary order, without maintaining any rollback state.

Efficient garbage collection in optimistic multi-writer database systems

Methods, systems, and computer-readable media for dynamic prefetching for database queries are disclosed. A query of a database is started according to a first prefetch policy. Before completing the query, the first prefetch policy is changed to a second prefetch policy. A portion of the query is performed according to the second prefetch policy.

Dynamic prefetching for database queries

Storage level query processing may be implemented for processing database queries. Nodes that can access a database may perform parallel processing for at least a portion of a database query. An indication may be received that specifies parallel processing for the database query. The nodes can then be caused to perform the portion of the query as part of providing a result in response to the database query instead of a node, such as a query engine node, that received the database query.

Storage level parallel query processing

A service provider network implements seamless scaling via proxy replay of session state. Upon a trigger, such as a determination to scale a server, a scaled server may be spun up and an identifier of the scaled server provided to a first (existing) server. The first server sends the identification of the second server, and session state information for each of the connections between the first server and the request router, to the request router. For each of the connections, the request router establishes a new connection between the request router and the second (scaled) server, and replays the session state information for the connection to the second server. The request router then routes traffic between each existing client connection (e.g., the same existing client connection which carried traffic delivered to the first server) and the corresponding new connection to the second server.

Seamless scaling via proxy replay of session state

Automatic consistency is implemented for forwarding writes in a database system. A read-only node may support both read and write requests to a database. The read only node may forward write requests to a read-write node. The read-only node may determine whether to include the forwarded requests in a version of the database for a read request automatically according to a consistency specified for a read request.

Automatic consistency for database write forwarding

Systems and methods are disclosed to implement, in a database system, a database engine node that generates write tracking transactions to atomically enforce cross-page constraints for write operations. In embodiments, the database system may include multiple database engine nodes that optimistically perform page writes without checking for potential conflicts at a shared storage system. The storage system is configured to detect and report conflicting writes on the same page. To cause the storage system to report cross-page conflicts for a write, the database engine node creates a transaction that includes the write and a write tracker (WT) operation for a tracked page. The WT operation will cause the storage system to report any conflicting writes to the tracked page before the actual write is accepted by the storage system. If a reported conflict is not favorably resolved, the database engine node will abort the entire transaction, including the actual write.

Atomic enforcement of cross-page data constraints in decoupled multi-writer databases

Batch processing of undo log records may be performed for rolling back a database transaction. When a rollback operation for a database transaction is performed, undo log records from an undo log to rollback the database transaction are divided into different groups of undo log records. Different requests for the different undo log records are sent to a storage system to be performed. If a conflict is detected for an undo log record in a group, an indication of the conflicting undo log record may be provided to indicate the conflict. The group of undo log records may be resent with a request to reapply the undo log records in the group. When all groups have been identified as successfully completed, then the rollback operation may be considered completed.

Batch undo processing for transaction rollbacks in a multi-writer database

Systems, methods, and computer storage media are disclosed. A computer storage medium includes instructions that, when executed, perform actions including maintaining a first view based on a first table of a database. The actions include performing a first transaction within the database. The actions include performing a second transaction within the database, and the second transaction relies on data from the first view. Performing the second transaction includes beginning the second transaction. In response to occurrence of both (i) the first transaction committing a first modification to the first table subsequent to the beginning of the second transaction and (ii) the second transaction causing a modification to the first view based on the first modification, performing the second transaction includes continuing the second transaction based on the data from the first view excluding the modification to the first view and completing the second transaction.

Reducing update conflicts when maintaining views

A storage engine may implement dynamic splitting of contentious data pages. Data pages may store data for a table of a data store as part of an indexing structure for the table. Access to the table may be provided by locating the corresponding data pages via the indexing structure. Access contention for different data pages may be monitored. Data pages may be identified for splitting based on the monitoring. A split operation for an identified data page may be formed to store the data on the identified data page on two different data pages so that subsequent access requests for the data are divided between the two data pages. Monitoring of access contention may also be performed to identify data pages for merging in order to consolidate access requests to a single data page.

Dynamic splitting of contentious index data pages

Systems and methods are disclosed to implement, in a database system, database engine nodes that implement multi-page writes as an atomic operation. In embodiments, the database system may include multiple database engine nodes that optimistically perform page writes locally without checking for potential conflicts at a shared storage system. The storage system will detect and report conflicting writes on a single page. To ensure that a group of writes to multiple pages occur atomically, the database engine node maintains a transaction context for the group, so that if a page conflict is later detected for any write in the group, the entire group will be rolled back at the database engine node and the storage system. In embodiments, the transaction context will track subsequent writes generated by the node to any page written by the group, so that these subsequent writes are also rolled back with the group.

Optimistic atomic multi-page write operations in decoupled multi-writer databases

Systems, methods, and computer-readable media are disclosed for an improved database. The systems, methods, and computer-readable media described herein may enhance the response time of databases and improve user experiences. In an example method described herein, a database may receive a search request for data stored in the database. The database may be stored in association with a b-tree to facilitate searching and retrieving data from the database. The method may include allocating a portion of a first memory for read ahead operations and performing the read ahead operations, wherein the read ahead operations comprise loading, into the portion of the first memory, data associated with one or more siblings of a current node, wherein the one or more siblings of the current node are one or more nodes that have a same number of parent nodes as the current node.

Methods and systems for an improved database

Growing community of inventors

Sammamish, WA, United States of America

Tengiz Kharatishvili