Amazon Technologies, Inc.

Xiaofeng Bao

Alexandre Olegovich Verbitski

Kamal Kant Gupta

Tengiz Kharatishvili

Niket Goel

Justin Jon Levandoski

Saileshwar Krishnamurthy

Murali Brahmadesam

Changan Han

Xiang Peng

Anurag Windlass Gupta

Raman Mittal

Saleem Mohideen

Punit Rajgaria

Seungmin Wei

Konstantin Dubinets

Tiffany Jianto

Venkatesh Nandakumar

Gopi K Attaluri

Yi Ding

Joseph Andrew Tucek

Bohan Liu

Li-Che Hsiao

Aakash Shah

Benjamin S Ellis

Zhaohui Zhang

Alexandre Olegovich Vebitski

Sheng-Liang Song

Zhonghua Feng

Ming Judy Lu

Alisdair William Owens

Kamal Kunt Gupta

Optimistic concurrency may be implemented for multiple writers to a database. Writes to a database from different writers may succeed unless a conflict is detected at the data store for the database. For detected conflicts, a request to resolve the conflict between writes may be submitted to a primary writer, which may determine and provide a resolution of the conflict to the conflicting writers. The data store may then be updated to commit the selected write identified by the resolution of the conflict.

Optimistic concurrency for a multi-writer database

Systems and methods are disclosed to implement, in a multi-writer database system, a database engine node that optimistically sends queued purge operations to a backend storage system without waiting for acknowledgement of previous operations from the storage system. If a conflict is later detected for a particular operation, the database engine node may retry some portion of the purge operations in the queue. In embodiments, the purge operations are performed as part of a garbage collection phase of a multi-version concurrency control (MVCC) protocol, where unneeded versions of objects previously generated for isolated snapshots are deleted. In embodiments, the operations in the queue may be grouped so that a rejection of one operation in the group will cause the entire group to be retried. Because these purge operations are idempotent and directed to objects that are invisible, they may be freely retried in arbitrary order, without maintaining any rollback state.

Efficient garbage collection in optimistic multi-writer database systems

Updates for transactions to a replication log for a database may be amortized. As updates are received as part of an active transaction, replication log records may be generated. When the size of replication log records exceeds a transaction size threshold, the replication log records may be stored in a remote data store instead of a local data store. When a request to commit the active transaction is received, an replication log for the database may be updated to include the replication log records from a transaction data object in the remote data store.

Amortizing replication log updates for transactions

Systems and methods are disclosed to implement, in a database system, a database engine node that generates write tracking transactions to atomically enforce cross-page constraints for write operations. In embodiments, the database system may include multiple database engine nodes that optimistically perform page writes without checking for potential conflicts at a shared storage system. The storage system is configured to detect and report conflicting writes on the same page. To cause the storage system to report cross-page conflicts for a write, the database engine node creates a transaction that includes the write and a write tracker (WT) operation for a tracked page. The WT operation will cause the storage system to report any conflicting writes to the tracked page before the actual write is accepted by the storage system. If a reported conflict is not favorably resolved, the database engine node will abort the entire transaction, including the actual write.

Atomic enforcement of cross-page data constraints in decoupled multi-writer databases

Batch processing of undo log records may be performed for rolling back a database transaction. When a rollback operation for a database transaction is performed, undo log records from an undo log to rollback the database transaction are divided into different groups of undo log records. Different requests for the different undo log records are sent to a storage system to be performed. If a conflict is detected for an undo log record in a group, an indication of the conflicting undo log record may be provided to indicate the conflict. The group of undo log records may be resent with a request to reapply the undo log records in the group. When all groups have been identified as successfully completed, then the rollback operation may be considered completed.

Batch undo processing for transaction rollbacks in a multi-writer database

Online restore operations for a database engine can be performed. A request to restore a database to a previous state can be received. Previously stored content of the database, such as snapshot stored prior to the previous state, can be identified along with log records describing changes to be made to the content prior to the previous state. State information in a query engine can be updated based on the previously stored content and log records so that queries can be processed based on the state information at the database restore to the previous state.

Online restore to a selectable prior state for database engines

Systems and methods are disclosed to implement, in a database system, database engine nodes that implement multi-page writes as an atomic operation. In embodiments, the database system may include multiple database engine nodes that optimistically perform page writes locally without checking for potential conflicts at a shared storage system. The storage system will detect and report conflicting writes on a single page. To ensure that a group of writes to multiple pages occur atomically, the database engine node maintains a transaction context for the group, so that if a page conflict is later detected for any write in the group, the entire group will be rolled back at the database engine node and the storage system. In embodiments, the transaction context will track subsequent writes generated by the node to any page written by the group, so that these subsequent writes are also rolled back with the group.

Optimistic atomic multi-page write operations in decoupled multi-writer databases

Online reordering for database table columns may allow a user to reorder the columns of a database table without taking the database offline. A mapping between different column orders for a database table may be generated and stored in response to a request to reorder columns in a database table. When a portion of the database table is accessed, the columns of the database table in the accessed portion of the database table may be reordered to perform the access request.

Online reordering of database table columns

Data pages of a database may be shared to create a copy of the database. A request to create a copy of the database may be received. Pointers to the data pages in the data store may be generated and stored as part of the copy of the database. When a request to read a data page in the copy of the database is received, the pointer to the data page may be accessed and the data page read from the database. When a request to modify a data page in the copy of the database is received, a modified version of the data page may be stored as part of the copy of the database and the pointer updated to point to the modified version of the data page.

Shared pages for database copies

Growing community of inventors

Fremont, CA, United States of America

Xiaofeng Bao