International Business Machines Corporation

Amazon Technologies, Inc.

Gopi K Attaluri

Vijayshankar Raman

Ronald Jason Barber

Richard Sefton Sidle

Liping Zhang

Kamal Kant Gupta

Ippokratis Pandis

David C Sharpe

Guy Maring Lohman

Saileshwar Krishnamurthy

Vincent Kulandaisamy

Alexandre Olegovich Verbitski

Yingjie He

Naresh K Chainani

Sam Sampson Lightstone

Tengiz Kharatishvili

Yongsik Yoon

David Joseph Wisneski

Garret Frederick Swart

Eugene Jon Shekita

Xun Xue

Lin Qiao

James Liam Finnie

Piotr M Plachta

Stewart L Palmer

Stefan Arndt

Dhruva Ranjan Chakrabarti

Atrayee Mullick

Roger L Q Zheng

Saraswathy Narayan

Damien Katz

Stefano Stefani

Anurag Windlass Gupta

Michael Jeffrey Winer

Yuri L Volobuev

Murali Brahmadesam

Xiaofeng Bao

Alok Nandan Nikhil

Niket Goel

Jeffrey Davis

Shriram Sridharan

Debanjan Saha

Aakash Shah

Francisco J Bermudez

Roger Lq Zheng

Robert Dean Jackson

Volodymyr Verovkin

Dhruva Ranjan Chakrabarit

Aleksandr Valerevich Feinberg

Namrata Bapat

Ian Kosseff

Harry Rohidas Nayak

Gerald A Fisher

Catalina Y Fan

Zhonghua Feng

Ming Judy Lu

Huawei Wang

Jianguo Wang

Storage-side filtering may be implemented for performing hash join operations with respect to data stored in distributed data storage. When a query to a database stored in a distributed data store is received, a hash join operation may be identified for performing the query. As part of performing the hash join operation, a query engine may cause storage nodes in the distributed data storing data for the database to filter data before sending the data to the query engine according to a join predicate for the hash join operation. A result of the query may then be provided to a user, using the filtered data provided from the storage nodes.

Storage-side filtering for performing hash joins

Transaction statements may be forwarded for a database transaction. A transaction begun at a read-only node may include a write statement. The read-only node may forward the write statement to the read-only node to perform as a transaction. The read-only node may update a read view for the transaction to include the write statement before performing a read statement submitted after the write statement at the read-only node.

Selective forwarding for multi-statement database transactions

Updates for transactions to a replication log for a database may be amortized. As updates are received as part of an active transaction, replication log records may be generated. When the size of replication log records exceeds a transaction size threshold, the replication log records may be stored in a remote data store instead of a local data store. When a request to commit the active transaction is received, an replication log for the database may be updated to include the replication log records from a transaction data object in the remote data store.

Amortizing replication log updates for transactions

Methods, systems, and computer-readable media for dynamic prefetching for database queries are disclosed. A query of a database is started according to a first prefetch policy. Before completing the query, the first prefetch policy is changed to a second prefetch policy. A portion of the query is performed according to the second prefetch policy.

Dynamic prefetching for database queries

Techniques for a context preservation during server restart are described. A restart of a database server having at least one open communications session with a database client is initiated. The database server can be executed by a compute instance hosted by a computer system of a provider network. A restarted database server is launched. The restarted database server obtains session state data to resume the at least one open communications session with the database client, the session state data including one or more session variables. The restarted database server receives a transaction from the database client.

Context preservation during server restart

Storage level query processing may be implemented for processing database queries. Nodes that can access a database may perform parallel processing for at least a portion of a database query. An indication may be received that specifies parallel processing for the database query. The nodes can then be caused to perform the portion of the query as part of providing a result in response to the database query instead of a node, such as a query engine node, that received the database query.

Storage level parallel query processing

A first table has more than one key column, and a hash array is built for the table, which includes computing hash indices for respective rows of the first table based on more than one key column. Building the hash array also includes writing in the hash array for each row at a respective location in the hash array, wherein each location is indicated by each row's respective hash index computed from the more than one key column. The hash array is probed responsive to the second table. In the building of the hash array for the first table, the writing in the hash array for each row at a respective location in the hash array writes only one of the key columns as a single key field in the hash array for each row.

Technology to reduce cost of concatenation for hash array

Automatic consistency is implemented for forwarding writes in a database system. A read-only node may support both read and write requests to a database. The read only node may forward write requests to a read-write node. The read-only node may determine whether to include the forwarded requests in a version of the database for a read request automatically according to a consistency specified for a read request.

Automatic consistency for database write forwarding

A hybrid query engine using different query engines may be implemented to provide access to a database. A database query to a database may be received at a hybrid query engine. A plan to perform the database query may be generated that when performed by the hybrid query engine causes one query engine to perform an assigned portion of the plan by accessing data that describes the database. Another portion of the plan assigned to another query engine may be performed by the hybrid query engine to cause the other query engine to perform the other portion of the plan by accessing the same data that describes the database.

Hybrid query execution engine using transaction and analytical engines

Transactional scans of portions of a database may be performed. When a query is received that causes a scan of a database, different portions of the database may be scanned within separate transactions. Operations applicable to perform the scan the records of the different portions of the database may be invoked once within the transactions instead of invoking the operations individually for each of the records in the different portions. Results of the query based on the scanned portions may then be provided in response to the query.

Transactional scanning of portions of a database

A method creating a compact hash table includes a thread executing using a processor for building a linear probe hash table based on allocating space for an array of keys. Payloads are inserted into the linear probe hash table. Keys from the array of keys are inserted into the linear probe hash table with linear probing. A linear probing bitmap is generated based on occupied locations in the linear probe hash table. A compact hash table is created based on the linear probe hash table and the linear probing bitmap.

Parallel build of non-partitioned join hash tables and non-enforced N:1 join hash tables

A computer program product provides efficient multiple aggregation distinct processing. The computer program product including a computer readable storage medium having program instructions embodied therewith. The program instructions are executable by a processor to cause the processor to convert a query with multiple aggregation distincts into a two-stage group-by process using a nest operator. The two-stage group-by process including further program instructions to cause the processor to: perform a first stage group-by process including the nest operator taking a single input row, and concatenating a grouping key with a measure value for each aggregation distinct that forms multiple inputs to a single group-by table, and perform a second stage group-by process including the nest operator bringing together entries for each original group.

Efficient multiple aggregation distinct processing

A method implemented by at least one processing device, a processing device, and a computer program product are provided for adjusting hash partitions for a hash join operation. An amount of rows within each of an initial number of hash partitions is determined during assignment of respective rows to respective hash partitions. The initial number of hash partitions is adjusted to a final number of hash partitions based on the amount of rows within each of the initial number of hash partitions and an amount of available memory. The adjusting further includes determining the final number of hash partitions based on the amount of rows within each of the initial number of hash partitions and the amount of available memory, and assigning the rows to the final number of hash partitions. The hash join operation is then performed.

Switching between a non-partitioned hash join and a partitioned hash join based on an amount of available memory

Filtered generation of hash tables may be implemented as part of performing a hash join. A database query may be received that causes a hash join to be performed in order to join two database tables. A filtered generated from a scan of one of the tables that indicates which data in the table can satisfy a join predicate for the hash join may be applied to generate a hash table from a portion of the other table. The filtered hash table may then be used to evaluate whether portions of the tables can be joined in satisfaction of the join predicate for the hash join.

Filtered hash table generation for performing hash joins

To manage memory in a multi-processing system, a memory budget is assigned to each of a number of agents within the multi-processing system. A portion of memory is allocated to each the agents within the memory budget. Metrics are collected for each agent during processing of data by the agents; the metrics include an amount of data processed and an amount of memory used for each agent. Memory efficiency is determined for each agent based on the collected metrics and another memory budget is determined based on the memory efficiency. The portion of the memory is reallocated to the agents within the other memory budget in response to data stored in the memory relative to the assigned memory budget meeting a criterion.

Memory management in multi-processor environments based on memory efficiency

Growing community of inventors

San Jose, CA, United States of America

Gopi K Attaluri