Amazon Technologies, Inc.

Kamal Kant Gupta

Saileshwar Krishnamurthy

Alexandre Olegovich Verbitski

Tengiz Kharatishvili

Anurag Windlass Gupta

Yongsik Yoon

Gopi K Attaluri

Stefano Stefani

Xiaofeng Bao

Alok Nandan Nikhil

Niket Goel

Bohan Liu

Justin Jon Levandoski

Yingjie He

Shriram Sridharan

Debanjan Saha

Aakash Shah

Mushahid Alam

Sangyong Hwang

Sandor Loren Maurice

Alexey Kuznetsov

Lawrence Douglas Webley

Sudipta Sengupta

Murali Brahmadesam

Joseph Andrew Tucek

Punit Rajgaria

Jaeyun Noh

Yuhui Yuan

Yoni Shalom

Jagdeep Singh Sidhu

Siyuan Xing

Yibo Zhu

Nicholas Piers O'Shea

Sangil Song

Hongbo Song

Zhaohui Zhang

Yan Valerie Leshinsky

Ippokratis Pandis

Yi Ding

Yuri L Volobuev

Michail Petropoulos

Eric Boutin

Saleem Mohideen

Changan Han

Jeffrey Davis

Damien Katz

Sailesh Krishnamurthy

Hyungsoo Jung

Xiang Peng

Tiffany Jianto

Sheng-Liang Song

Madhura Madhukar Parikh

Alexandre Olegovich Vebitski

Volodymyr Verovkin

Dhruva Ranjan Chakrabarit

Aleksandr Valerevich Feinberg

Namrata Bapat

Ian Kosseff

Pallavi Maheshwara Kakunje

Zhonghua Feng

Ming Judy Lu

Storage-side filtering may be implemented for performing hash join operations with respect to data stored in distributed data storage. When a query to a database stored in a distributed data store is received, a hash join operation may be identified for performing the query. As part of performing the hash join operation, a query engine may cause storage nodes in the distributed data storing data for the database to filter data before sending the data to the query engine according to a join predicate for the hash join operation. A result of the query may then be provided to a user, using the filtered data provided from the storage nodes.

Storage-side filtering for performing hash joins

Proxy-based scaling may be performed for databases. A proxy may be implemented for a database that can establish a connection between the proxy and a database engine to perform a database queries received from a client at the proxy. A scaling event may be detected for the database responsive to which the proxy may establish a connection with a new database engine which may, in some embodiments, have different capabilities or resources that address the features or criteria that triggered the scaling event. Session state may be copied from the database engine to the new database engine so that the new database engine may be able to provide access to the database on behalf of requests received from the client through the proxy.

Proxy-based database scaling

Optimistic concurrency may be implemented for multiple writers to a database. Writes to a database from different writers may succeed unless a conflict is detected at the data store for the database. For detected conflicts, a request to resolve the conflict between writes may be submitted to a primary writer, which may determine and provide a resolution of the conflict to the conflicting writers. The data store may then be updated to commit the selected write identified by the resolution of the conflict.

Optimistic concurrency for a multi-writer database

Change data capture is performed asynchronously for direct transfer to a stream processor. When updates to a database as part of database transactions are performed, a transaction log is generated that describes the database transactions. The transaction log may be evaluated to determine indications of changes included in the database transactions. The indications of the changes may be stored in memory according to the database transactions that include the indications of the changes. When a database transaction is determined to be committed to the database, the change indications may be sent directly from the memory to the stream processor.

Asynchronous change data capture for direct external transmission

Queries to a consistent view of a data set across different types of query engines are performed. Portions of a query received at a first type of query engine that can be performed at a different type of a query engine can be identified. A consistent view of a data set stored in a common data store can be provided by the first type of query engine to the second query engine. When the different type of query engine performs the portion of the query, the different type of query engine applies the consistent view of the data set. A result for the query can be returned by the first type of query engine based on results from the different type of query engine.

Performing queries to a consistent view of a data set across query engine types

Updates for transactions to a replication log for a database may be amortized. As updates are received as part of an active transaction, replication log records may be generated. When the size of replication log records exceeds a transaction size threshold, the replication log records may be stored in a remote data store instead of a local data store. When a request to commit the active transaction is received, an replication log for the database may be updated to include the replication log records from a transaction data object in the remote data store.

Amortizing replication log updates for transactions

Methods, systems, and computer-readable media for dynamic prefetching for database queries are disclosed. A query of a database is started according to a first prefetch policy. Before completing the query, the first prefetch policy is changed to a second prefetch policy. A portion of the query is performed according to the second prefetch policy.

Dynamic prefetching for database queries

Techniques for making machine learning inference calls for database query processing are described. In some embodiments, a method of making machine learning inference calls for database query processing may include generating a first batch of machine learning requests based at least on a query to be performed on data stored in a database service, wherein the query identifies a machine learning service, sending the first batch of machine learning requests to an input buffer of an asynchronous request handler, the asynchronous request handler to generate a second batch of machine learning requests based on the first batch of machine learning requests, and obtaining a plurality of machine learning responses from an output buffer of the asynchronous request handler, the machine learning responses generated by the machine learning service using a machine learning model in response to receiving the second batch of machine learning requests.

Machine learning inference calls for database query processing

Techniques for a context preservation during server restart are described. A restart of a database server having at least one open communications session with a database client is initiated. The database server can be executed by a compute instance hosted by a computer system of a provider network. A restarted database server is launched. The restarted database server obtains session state data to resume the at least one open communications session with the database client, the session state data including one or more session variables. The restarted database server receives a transaction from the database client.

Context preservation during server restart

Storage level query processing may be implemented for processing database queries. Nodes that can access a database may perform parallel processing for at least a portion of a database query. An indication may be received that specifies parallel processing for the database query. The nodes can then be caused to perform the portion of the query as part of providing a result in response to the database query instead of a node, such as a query engine node, that received the database query.

Storage level parallel query processing

A service provider network implements seamless scaling via proxy replay of session state. Upon a trigger, such as a determination to scale a server, a scaled server may be spun up and an identifier of the scaled server provided to a first (existing) server. The first server sends the identification of the second server, and session state information for each of the connections between the first server and the request router, to the request router. For each of the connections, the request router establishes a new connection between the request router and the second (scaled) server, and replays the session state information for the connection to the second server. The request router then routes traffic between each existing client connection (e.g., the same existing client connection which carried traffic delivered to the first server) and the corresponding new connection to the second server.

Seamless scaling via proxy replay of session state

Alternative portions of a query plan to perform a query may be selected for processing partial results of a query. A query engine may generate a plan to perform a query that includes an operation that selects between different portions of the plan according to input data. The input data may be a partial result of the query generated by the data store instead of the query engine and provided to the query engine. The input data may be an individual item of the data set that is modified to a different version within a consistent view of the data set for the query.

Selecting alternate portions of a query plan for processing partial results generated separate from a query engine

Systems and methods are disclosed to implement, in a database system, a database engine node that generates write tracking transactions to atomically enforce cross-page constraints for write operations. In embodiments, the database system may include multiple database engine nodes that optimistically perform page writes without checking for potential conflicts at a shared storage system. The storage system is configured to detect and report conflicting writes on the same page. To cause the storage system to report cross-page conflicts for a write, the database engine node creates a transaction that includes the write and a write tracker (WT) operation for a tracked page. The WT operation will cause the storage system to report any conflicting writes to the tracked page before the actual write is accepted by the storage system. If a reported conflict is not favorably resolved, the database engine node will abort the entire transaction, including the actual write.

Atomic enforcement of cross-page data constraints in decoupled multi-writer databases

Growing community of inventors

Snoqualmie, WA, United States of America

Kamal Kant Gupta