Oracle International Corporation

Hewlett Packard Enterprise Development LP

Microsoft Technology Licensing, LLC

Hideaki Kimura

Garret Frederick Swart

Juan R Loaiza

Tirthankar Lahiri

Jesse Kamp

Avneesh Pant

Spyros Blanas

Tianzheng Wang

Meichun Hsu

Shasank Kisan Chavan

Yunrui Li

Boris Klots

Allison Holloway

Ajit Mylavarapu

Teck Hua Lee

Vasudha Krishnaswamy

Aurosish Mishra

Sukhada Pendse

Kartik Kulkarni

Milind M Chabbi

Shuo Yang

Tinggang Wang

Vivek Ravindranath Narasayya

Goetz Graefe

Harumi Kuno

Alan D Halverson

Manoj A Syamala

Nikhil Teletia

Srinath Shankar

Willis Lang

Karthik Saligrama Ramachandra

Allison L Holloway

A shared-nothing database system is provided in which parallelism and workload balancing are increased by assigning the rows of each table to 'slices', and storing multiple copies (“duplicas”) of each slice across the persistent storage of multiple nodes of the shared-nothing database system. When the data for a table is distributed among the nodes of a shared-nothing system in this manner, requests to read data from a particular row of the table may be handled by any node that stores a duplica of the slice to which the row is assigned. For each slice, a single duplica of the slice is designated as the “primary duplica”. All DML operations (e.g. inserts, deletes, updates, etc.) that target a particular row of the table are performed by the node that has the primary duplica of the slice to which the particular row is assigned. The changes made by the DML operations are then propagated from the primary duplica to the other duplicas (“secondary duplicas”) of the same slice.

System and method for an ultra highly available, high performance, persistent memory optimized, scale-out database

Systems and methods for reducing latency of probing operations of remotely located linear hash tables are described herein. In an embodiment, a system receives a request to perform a probing operation on a remotely located linear hash table based on a key value. Prior to performing the probing operation, the system dynamically predicts a number of slots for a single read of the linear hash table to minimize total cost for an average probing operation. The system determines a hash value based on the key value and determines a slot of the linear hash table to which the hash value corresponds. After predicting the number of slots, the system issues an RDMA request to perform a read of the predicted number of slots from the linear hash table starting at the slot to which the hash value corresponds.

Efficient usage of one-sided RDMA for linear probing

In example implementations, a method include receiving a request for a lock in a Mellor-Crummey Scott (MCS) lock protocol from a guest user that is context free (e.g., a process that does not bring a queue node). The lock determines that it contains a null value. The lock is granted to the guest user. A pi value is received from the guest user to store in the lock. The pi value notifies subsequent users that the guest user has the lock.

Request of an MCS lock by guests

Techniques are described for offloading remote direct memory operations (RDMOs) to 'execution candidates'. The execution candidates may be any hardware capable of performing the offloaded operation. Thus, the execution candidates may be network interface controllers, specialized co-processors, FPGAs, etc. The execution candidates may be on a machine that is remote from the processor that is offloading the operation, or may be on the same machine as the processor that is offloading the operation. Details for certain specific RDMOs, which are particularly useful in online transaction processing (OLTP) and hybrid transactional/analytical (HTAP) workloads, are provided.

Same-machine remote direct memory operations (RDMOS)

A shared-nothing database system is provided in which parallelism and workload balancing are increased by assigning the rows of each table to 'slices', and storing multiple copies ('duplicas') of each slice across the persistent storage of multiple nodes of the shared-nothing database system. When the data for a table is distributed among the nodes of a shared-nothing system in this manner, requests to read data from a particular row of the table may be handled by any node that stores a duplica of the slice to which the row is assigned. For each slice, a single duplica of the slice is designated as the &#x201c;primary duplica&#x201d;. All DML operations (e.g. inserts, deletes, updates, etc.) that target a particular row of the table are performed by the node that has the primary duplica of the slice to which the particular row is assigned. The changes made by the DML operations are then propagated from the primary duplica to the other duplicas (&#x201c;secondary duplicas&#x201d;) of the same slice.

Techniques are provided to allow more sophisticated operations to be performed remotely by machines that are not fully functional. Operations that can be performed reliably by a machine that has experienced a hardware and/or software error are referred to herein as Remote Direct Memory Operations or 'RDMOs'. Unlike RDMAs, which typically involve trivially simple operations such as the retrieval of a single value from the memory of a remote machine, RDMOs may be arbitrarily complex. The techniques described herein can help applications run without interruption when there are software faults or glitches on a remote system with which they interact.

One-sided reliable remote direct memory operations

Distributed query processing is often performed by a set of nodes that apply MapReduce to a data set and materialize partial results to storage, which are then aggregated to produce the query result. However, this architecture requires a preconfigured set of database nodes; can only fulfill queries that utilize MapReduce processing; and may be slowed down by materializing partial results to storage. Instead, distributed query processing can be achieved by choosing a node for various portions of the query, and generating customized code for the node that only performs the query portion that is allocated to the node. The node executes the code to perform the query portion, and rather than materializing partial results to storage, streams intermediate query results to a next selected node in the distributed query. Nodes selection may be involve matching the details of the query portion with the characteristics and capabilities of the available nodes.

Per-node custom code engine for distributed query processing

Example implementations disclosed herein can be used to build, maintain, and access databases built database in multi-core computing systems with large VRAM and huge NVRAM. The database with optimistic concurrency control can be built on a transactional key-value data store that includes logically equivalent data pages stored in both VRAM and VRAM. Data records in volatile data pages in the VRAM represent the most recent version of the data. Data records in the NVRAM immutable and are organized in a stratified composite snapshot. A distributed log gleaner process is used to process log entries corresponding to transactions on the volatile data pages and construct the snapshot. The log gleaner sorts the log entries by epoch, key range, and most recent use to partition the snapshot across multiple nodes.

Transactional key-value store

Example implementations disclosed herein include techniques for a ready only bufferpool for use in local nodes of a multi-node computing system. Read only transactions executed by a processor can reference a ready only bufferpool resident in a VRAM on the same node. If the desired data page is in the bufferpool the transaction can immediately read data records from the cached data pages. If the desired data page is not in the bufferpool, then the transaction can cause a copy of a corresponding data page in a secondary memory to be installed in the bufferpool. The bufferpool can include more than one copy of a data page simultaneously to handle and prevent cache line misses. Data page are dropped from the bufferpool based on an incrementing per data page counter.

Read only bufferpool

Hybrid concurrency control in a database transaction is disclosed. A contention value associated with a database object is read. The contention value is dynamically updated based on access patterns of the database object. If the contention value indicates the database object is under high contention, a pessimistic concurrency control mechanism is applied to the database object. An optimistic concurrency control mechanism is applied to the database object if the contention value does not indicate the database object is under high contention.

Hybrid concurrency control

Example implementations disclosed herein can be used to build, maintain, and use a hash table distributed across the plurality multiple nodes in a multi-node computing system. The hash table can include data pages associated by corresponding pointers according to a tree data structure. The data pages include leaf data pages. Each leaf data page can be associated with a corresponding hash value and include a tag bitmap. When a transaction associated with a key is executed, a hash value and a tag value are generated based on the key. The leaf data pages can be searched using the hash value. A probability that a leaf data page includes the key can be determined based on a comparison tag value with the tag bitmap.

Hash index

Growing community of inventors

Palo Alto, CA, United States of America

Hideaki Kimura