Amazon Technologies, Inc.

Microsoft Technology Licensing, LLC

Alexandre Olegovich Verbitski

Kamal Kant Gupta

Tengiz Kharatishvili

Anurag Windlass Gupta

Saileshwar Krishnamurthy

Gopi K Attaluri

Xiaofeng Bao

Alok Nandan Nikhil

Niket Goel

Stefano Stefani

Bohan Liu

Saleem Mohideen

David Charles Wein

Shriram Sridharan

Debanjan Saha

Mushahid Alam

Dhruva Ranjan Chakrabarti

Justin Jon Levandoski

Murali Brahmadesam

Atrayee Mullick

Sandor Loren Maurice

Yingjie He

Alexey Kuznetsov

Lawrence Douglas Webley

Gaurav Kumar Gupta

Anand Kumar Thakur

Upendra Govindagowda

Saraswathy Narayan

Damien Katz

Hyungsoo Jung

Tomas Talius

Dexter Paul Bradshaw

Joseph Andrew Tucek

Chengliang Zhang

Punit Rajgaria

Pedro Celis

Georgiy Reynya

Santeri Olavi Voutilainen

Changan Han

Navaneetha Krishnan Thanka Nadar

Andrew Scott Richardson

Jagdeep Singh Sidhu

Yuhui Yuan

Aakash Shah

Yoni Shalom

Hongbo Song

Sadashivan Krishnamurthy

Boris Baryshnikov

Siyuan Xing

Yibo Zhu

Tiffany Jianto

James Laurence Finnerty

Seungmin Wei

Sean Oczkowski

Nicholas Piers O'Shea

Michael Nelson

Yan Valerie Leshinsky

Ajay Kalhan

Yi Ding

Conor J Cunningham

Yuri L Volobuev

Aleksandras Surna

Raman Mittal

Vishrut Shah

Eric Boutin

Ming-Chuan Wu

Jeffrey Davis

Hao Xia

Yixue Zhu

Fabricio Voznika

Jeffrey A East

Grant Alexander McAlister

Pravin Mittal

Baskar Durairaj

Xiang Peng

Sailesh Krishnamurthy

In-Jerng Choe

Apoorv Birthare

Clifford T Dibble

Benjamin S Ellis

Konstantin Dubinets

Jin Chen

Venkatesh Nandakumar

Haritabh Gupta

Ahmad Mohammad Radi Ahmad Alsmair

Gajanan Sharadchandra Chinchwadkar

Jan Engelsberg

Zhaohui Zhang

Pallavi Maheshwara Kakunje

Sree Kuchibhotla

Yuxi Bai

Ravishankar Rajamani

Alisdair William Owens

Volodymyr Verovkin

Dhruva Ranjan Chakrabarit

Aleksandr Valerevich Feinberg

Namrata Bapat

Ian Kosseff

Suhas Keshavamurthy

Clock times may be stored at nodes of a database system that are used to provide access to database data using Multi-Version Concurrency Control (MVCC) and implement isolation for distributed transactions. When an access request is received at a processing node of a database, the snapshot time of the access request is determined. The snapshot may be a range of time, with an earliest and latest timestamp. If the snapshot time has a latest time stamp greater than a current timestamp in the clock time, then the clock time may be updated. The clock time and snapshot time may be used to implement time-based MVCC to select a version of a tuple stored in the database to use for performing the access request.

Increasing commit times at database shards for implementing consistency and isolation for distributed transactions

Distributed execution of transactions may be performed across shards of a scalable database table. Instructions to perform as part of a transaction with respect to one or more database tables may be received. The instructions may be evaluated with respect to metadata obtained for the database tables to determine an assignment distribution of computing resources of the database system to data that can satisfy the access request. The commit protocol for the transaction may be determined according to the assignment distribution of the computing resources. After a request to commit the transaction is received, the transaction may be committed according to the determined commit protocol.

Distributed transaction execution across shards of scalable database tables

Proxy-based scaling may be performed for databases. A proxy may be implemented for a database that can establish a connection between the proxy and a database engine to perform a database queries received from a client at the proxy. A scaling event may be detected for the database responsive to which the proxy may establish a connection with a new database engine which may, in some embodiments, have different capabilities or resources that address the features or criteria that triggered the scaling event. Session state may be copied from the database engine to the new database engine so that the new database engine may be able to provide access to the database on behalf of requests received from the client through the proxy.

Proxy-based database scaling

Transaction statements may be forwarded for a database transaction. A transaction begun at a read-only node may include a write statement. The read-only node may forward the write statement to the read-only node to perform as a transaction. The read-only node may update a read view for the transaction to include the write statement before performing a read statement submitted after the write statement at the read-only node.

Selective forwarding for multi-statement database transactions

Intelligent query routing may be performed across shards of a scalable database table. A router of a database system may receive an access request directed to one or more database tables. The router may evaluate the access request with respect to metadata obtained for the database tables to determine an assignment distribution of computing resources of the database system to data that can satisfy the access request. The router can select planning locations to perform the access request based on the assignment distribution of the computing resources. The router can cause the access request to be performed according to planning at the selected planning locations.

Intelligent query routing across shards of scalable database tables

Lease-based consistency may be implemented for databases to handle failovers. A database node may obtain a consistency lease that describes a point in time determined from a time-to-live amount added to a consistent point in time for database data. While the consistency lease is valid, Multi-version Concurrency Control (MVCC) snapshots assigned by the database node can be used to handle requests to access the database data. Once expired, the database node may have to renew the consistency lease in order to continue to handle write and read requests.

Lease-based consistency management for handling failover in a database

Optimistic concurrency may be implemented for multiple writers to a database. Writes to a database from different writers may succeed unless a conflict is detected at the data store for the database. For detected conflicts, a request to resolve the conflict between writes may be submitted to a primary writer, which may determine and provide a resolution of the conflict to the conflicting writers. The data store may then be updated to commit the selected write identified by the resolution of the conflict.

Optimistic concurrency for a multi-writer database

Systems and methods are disclosed to implement, in a multi-writer database system, a database engine node that optimistically sends queued purge operations to a backend storage system without waiting for acknowledgement of previous operations from the storage system. If a conflict is later detected for a particular operation, the database engine node may retry some portion of the purge operations in the queue. In embodiments, the purge operations are performed as part of a garbage collection phase of a multi-version concurrency control (MVCC) protocol, where unneeded versions of objects previously generated for isolated snapshots are deleted. In embodiments, the operations in the queue may be grouped so that a rejection of one operation in the group will cause the entire group to be retried. Because these purge operations are idempotent and directed to objects that are invisible, they may be freely retried in arbitrary order, without maintaining any rollback state.

Efficient garbage collection in optimistic multi-writer database systems

Updates for transactions to a replication log for a database may be amortized. As updates are received as part of an active transaction, replication log records may be generated. When the size of replication log records exceeds a transaction size threshold, the replication log records may be stored in a remote data store instead of a local data store. When a request to commit the active transaction is received, an replication log for the database may be updated to include the replication log records from a transaction data object in the remote data store.

Amortizing replication log updates for transactions

Database commands may be asynchronously forwarded between nodes of a database system. A read-only node may receive a request, such as a read request or write request, and determine that the request should be forwarded to a read-only node for performance. A thread that handles the request may forward the request to the read-only node and register a call-back event for a response to the request and be made available to perform other work. When the response from the read-write node is received, the call-back event may be triggered and return a response to the request using the same or different thread.

Asynchronously forwarding database commands

Methods, systems, and computer-readable media for dynamic prefetching for database queries are disclosed. A query of a database is started according to a first prefetch policy. Before completing the query, the first prefetch policy is changed to a second prefetch policy. A portion of the query is performed according to the second prefetch policy.

Dynamic prefetching for database queries

Techniques for a context preservation during server restart are described. A restart of a database server having at least one open communications session with a database client is initiated. The database server can be executed by a compute instance hosted by a computer system of a provider network. A restarted database server is launched. The restarted database server obtains session state data to resume the at least one open communications session with the database client, the session state data including one or more session variables. The restarted database server receives a transaction from the database client.

Context preservation during server restart

Storage level query processing may be implemented for processing database queries. Nodes that can access a database may perform parallel processing for at least a portion of a database query. An indication may be received that specifies parallel processing for the database query. The nodes can then be caused to perform the portion of the query as part of providing a result in response to the database query instead of a node, such as a query engine node, that received the database query.

Storage level parallel query processing

A service provider network implements seamless scaling via proxy replay of session state. Upon a trigger, such as a determination to scale a server, a scaled server may be spun up and an identifier of the scaled server provided to a first (existing) server. The first server sends the identification of the second server, and session state information for each of the connections between the first server and the request router, to the request router. For each of the connections, the request router establishes a new connection between the request router and the second (scaled) server, and replays the session state information for the connection to the second server. The request router then routes traffic between each existing client connection (e.g., the same existing client connection which carried traffic delivered to the first server) and the corresponding new connection to the second server.

Seamless scaling via proxy replay of session state

Growing community of inventors

Woodinville, WA, United States of America

Alexandre Olegovich Verbitski