Amazon Technologies, Inc.

Punit Rajgaria

Murali Brahmadesam

Hemanth Satyanarayana

Alexandre Olegovich Verbitski

Gokul Soundararajan

Kamal Kant Gupta

Tengiz Kharatishvili

Yuri L Volobuev

Raman Mittal

Xiaofeng Bao

Changan Han

Niket Goel

Sanuj Basu

Li-Che Hsiao

Gopal Paliwal

Ramesh Chander

Anurag Windlass Gupta

Saileshwar Krishnamurthy

Ippokratis Pandis

Gaurav Saxena

Yi Ding

Justin Jon Levandoski

Sangyong Hwang

Joseph Andrew Tucek

Saleem Mohideen

Matthew Perry Abrams

Kartikey Bhatt

Bohan Liu

Nanda Kaushik

Todd Jeffrey Green

Tiffany Jianto

Vadim Skipin

Michael Higgins Dowling

Omar Farhat

Aakash Ashwin Shah

Johannes Wust

Techniques for fast database recovery in a multi-volume database environment via transactional awareness are described. In the event of a failure associated with a first volume storing database page data, the first volume can be restored to a point in time and transactional metadata from a second volume storing logical change data can be obtained for a limited number of transactions occurring at/after that point in time, as opposed to analyzing extremely large change log files. These transactions can be checked to ensure that they have all been persisted, and if not, change data for those transactions can be obtained from the second volume and used to replay these transactions on the restored first volume.

Fast database recovery in a multi-volume database environment via transactional awareness

Techniques for volume durable log sequence number movement in a multi-volume database environment are described. Blocks of database updates are persisted in an atomic and durable manner, where the blocks may include updates to a first volume, a second volume, or both. Backlinks from one block of updates to the previous block of updates may be set, in addition to backlinks between updates to the first volume and backlinks between updates to the second volume. Upon durably persisting a block, the block backlinks can be followed to help verify that all contiguous blocks—of changes to one or both volumes—have all been persisted. Thereafter, a volume durable log sequence number (VDL) pointer value can be updated to point to a known 'last' durably persisted log update.

Volume durable log sequence number movement in a multi-volume database environment

Methods for replicating transactional tables of a transactional database to an analytical database and maintaining updates to those transactional table representations are disclosed. Snapshots of the transactional tables are provided to the analytical database via a transport mechanism, such as a data storage service or a data streaming service, and stored at the analytical database. Then, checkpoints comprising portions of a change-data-capture log that has recorded transactional changes to the transactional tables of the transactional database are provided to the analytical database via the same or different transport mechanism and used to commit those transactional changes to the snapshot representations. The snapshot representations may be used to respond to incoming analytical queries in order to provide real-time querying results.

Hybrid transactional and analytical processing architecture for optimization of real-time analytical querying

Methods for negotiating parameters for maintaining, at an analytical database, a replica of a transactional table, stored at a transactional database, are disclosed. By establishing parameters of said maintenance prior to commencing, the transactional database and the analytical database may remain only loosely coupled after completing the negotiation. The parameters of the negotiation may be stored in a location that is accessible to both the transactional database and the analytical database, such that the results of the negotiation may be accessed during the process of maintaining the replica. The negotiation may include a procedure to provide portions of a change-data-capture log, associated to the transactional table, to the analytical database via a transport mechanism such that the replica may be updated, in addition to a list of agreed upon data definition language commands that may be used in the portions of the change-data-capture log.

Handshake protocol for efficient exchange of transactional information for a hybrid transactional and analytical processing architecture

Techniques for change data capture (CDC) log augmentation are described. In some examples, a user configures CDC log augmentation by indicating which data should be included in a CDC log, and the database, when generating a CDC log associated with this configuration, can obtain the associated data and augment the CDC log by inserting this data into it. The augmented data can include one or more fields from a record in a separate database table, where the record can be identified based on the changed record represented by the CDC log.

Change data capture log augmentation for streamlined ETL processing

Optimistic concurrency may be implemented for multiple writers to a database. Writes to a database from different writers may succeed unless a conflict is detected at the data store for the database. For detected conflicts, a request to resolve the conflict between writes may be submitted to a primary writer, which may determine and provide a resolution of the conflict to the conflicting writers. The data store may then be updated to commit the selected write identified by the resolution of the conflict.

Optimistic concurrency for a multi-writer database

Direct storage loading may be used to add data to a database. New data may be added to a database, using nodes different than a database engine to access a database. The addition of the new data may be assigned to different nodes. The nodes may obtain the data and store the data to storage locations according allocated space in the database by the database engine. The new data can then be made available for access at the database engine.

Direct storage loading for adding data to a database

Techniques for distributed database storage utilizing split volume types for redo log records and change log records are described. A database engine sends redo log data to a first set of page storage nodes and change log data to a second set of tuple storage nodes, where the change log data is stored in an indexed manner Client access to change log data may be provided through a separate set of streaming storage servers instead of through the database engine.

Streaming database change data from distributed storage

Systems and methods are disclosed to implement, in a database system, a database engine node that generates write tracking transactions to atomically enforce cross-page constraints for write operations. In embodiments, the database system may include multiple database engine nodes that optimistically perform page writes without checking for potential conflicts at a shared storage system. The storage system is configured to detect and report conflicting writes on the same page. To cause the storage system to report cross-page conflicts for a write, the database engine node creates a transaction that includes the write and a write tracker (WT) operation for a tracked page. The WT operation will cause the storage system to report any conflicting writes to the tracked page before the actual write is accepted by the storage system. If a reported conflict is not favorably resolved, the database engine node will abort the entire transaction, including the actual write.

Atomic enforcement of cross-page data constraints in decoupled multi-writer databases

Growing community of inventors

Sunnyvale, CA, United States of America

Punit Rajgaria