Oracle International Corporation

Allison Holloway

Shasank Kisan Chavan

Jesse Kamp

Vineet Marwah

Amit Ganesh

Michael J Gleeson

Tirthankar Lahiri

Anindya Chandra Patthak

Niloy Mukherjee

Teck Hua Lee

Sanket Hase

Roger MacNicol

Dennis Lui

Juan R Loaiza

Manosiz Bhattacharyya

Lawrence J Ellison

Vivekanandhan Raja

Aurosish Mishra

Krishna Kunchithapadam

Sheldon A K Lewis

Meichun Hsu

Garret Frederick Swart

Yunrui Li

Boris Klots

Ajit Mylavarapu

Vasudha Krishnaswamy

Hideaki Kimura

Sukhada Pendse

Kartik Kulkarni

Zhen Hua Liu

Sujatha Muthulingam

Ramesh Kumar

Atrayee Mullick

Xia Hua

Adrian Ng

Chinmayi Krishnappa

Roger Dermot Macnicol

Maria Colgan

Vicente Hernandez

Maheswaran Venkatachalam

Roger Macnicol

A shared-nothing database system is provided in which parallelism and workload balancing are increased by assigning the rows of each table to 'slices', and storing multiple copies (“duplicas”) of each slice across the persistent storage of multiple nodes of the shared-nothing database system. When the data for a table is distributed among the nodes of a shared-nothing system in this manner, requests to read data from a particular row of the table may be handled by any node that stores a duplica of the slice to which the row is assigned. For each slice, a single duplica of the slice is designated as the “primary duplica”. All DML operations (e.g. inserts, deletes, updates, etc.) that target a particular row of the table are performed by the node that has the primary duplica of the slice to which the particular row is assigned. The changes made by the DML operations are then propagated from the primary duplica to the other duplicas (“secondary duplicas”) of the same slice.

System and method for an ultra highly available, high performance, persistent memory optimized, scale-out database

Techniques for processing top-K queries are provided. In one technique, a database statement is received that requests top-K results related to a database object and that indicates two columns thereof: a first column by which to partition a result set and a second column by which to order the result set. A buffer is generated. For each of multiple rows in the database object: a first key value that associated with a first value in the first column of said each row is identified; a second key value that associated with a second value in the second column of said each entry is identified; a slot in the buffer is identified based on the first key value and the second key value; and the slot in the buffer may be updated based on the second key value. A response to the database statement is generated based on the buffer.

Processing top-K queries on data in relational database systems

Techniques herein use in-memory column vectors to process data that is external to a database management system (DBMS) and logically join the external data with data that is native to the DBMS. In an embodiment, a computer maintains a data dictionary for native data that is durably stored in an DBMS and external data that is not durably stored in the DBMS. From a client through a connection to the DBMS, the computer receives a query. The computer loads the external data into an in-memory column vector that resides in random access memory of the DBMS. Based on the query and the data dictionary, the DBMS executes a data join of the in-memory column vector with the native data. To the client through said connection, the computer returns results of the query based on the data join.

Performing in-memory columnar analytic queries on externally resident data

A shared-nothing database system is provided in which parallelism and workload balancing are increased by assigning the rows of each table to 'slices', and storing multiple copies ('duplicas') of each slice across the persistent storage of multiple nodes of the shared-nothing database system. When the data for a table is distributed among the nodes of a shared-nothing system in this manner, requests to read data from a particular row of the table may be handled by any node that stores a duplica of the slice to which the row is assigned. For each slice, a single duplica of the slice is designated as the &#x201c;primary duplica&#x201d;. All DML operations (e.g. inserts, deletes, updates, etc.) that target a particular row of the table are performed by the node that has the primary duplica of the slice to which the particular row is assigned. The changes made by the DML operations are then propagated from the primary duplica to the other duplicas (&#x201c;secondary duplicas&#x201d;) of the same slice.

Techniques are described herein for hybrid parallelization of in-memory table scans. Work for an in-memory scan is divided into granules based on a degree of parallelism. The granules are assigned to one or more processes. The work for each granule is further parallelized by dividing the work granule into one or more tasks. The tasks are assigned to one or more threads, the number of which can be dynamically adjusted.

Efficient hybrid parallelization for in-memory scans

Techniques related to efficient evaluation of query expressions including grouping clauses are disclosed. Computing device(s) perform a method for aggregating a measure column vector (MCV) according to a plurality of grouping column vectors (GCVs). The MCV and each of the plurality of GCVs may be encoded. The method includes determining a plurality of actual grouping keys (AGKs) and generating a dense identifier (DI) mapping that maps the plurality of AGKs to a plurality of DIs. Each AGK occurs in the plurality of GCVs. Each DI corresponds to a respective workspace. Aggregating the MCV involves aggregating, in each workspace, one or more codes of the MCV that correspond to an AGK mapped to a DI corresponding to the workspace. For a first row of the MCV and the plurality of GCVs, aggregating the one or more codes includes generating a particular grouping key based on codes in the plurality of GCVs.

Efficient evaluation of query expressions including grouping clauses

Techniques related to efficient evaluation of queries with multiple predicate expressions are disclosed. A first predicate expression (PE) is evaluated against a plurality of rows in a first column vector (CV) to determine that a subset of rows does not satisfy the first PE. The subset comprises less than all of the plurality of rows. When a query specifies the first PE in conjunction with a second PE, a selectivity of the first PE is determined. If the selectivity meets a threshold, the second PE is evaluated against all of the plurality of rows in a second CV. If the selectivity does not meet the threshold, the second PE is evaluated against only the subset of rows in the second CV. When a query specifies the first PE in disjunction with the second PE, the second PE may be evaluated against only the subset of rows in the second CV.

Efficient evaluation of queries with multiple predicate expressions

Techniques related to efficient evaluation of aggregate functions are disclosed. Computing device(s) may perform a method for aggregating results of performing a multiplication on a first column and a second column of a database table. A first vector stores a subset of values of the first column. A second vector stores a corresponding subset of values of the second column. When it is determined that the first vector has a lower cardinality than the second vector, a third vector stores at least a first distinct value and a second distinct value of the first vector. A first set of one or more values of the second vector is determined, wherein each value of the first set of one or more values corresponds to the first distinct value in the first vector. A first multiplicand is generated based on performing a summation over the first set of one or more values.

Efficient evaluation of aggregate functions

A method for accelerating queries using dynamically generated columnar data in a flash cache is provided. In an embodiment, a method comprises a storage device receiving a first request for data that is stored in the storage device in a base major format in one or more primary storage devices. The storage device comprises a cache. The base major format is any one of: a row-major format, a column-major format and a hybrid-columnar format. Based on first one or more criteria, it is determined whether to rewrite the data into rewritten data in a rewritten major format. In response to determining to rewrite the data into rewritten data in a rewritten major format, the storage device rewrites at least a portion of the data into particular rewritten data in the rewritten major format. The rewritten data is stored in the cache.

Method to accelerate queries using dynamically generated alternate data formats in flash cache

A method and apparatus for efficiently processing data in various formats in a single instruction multiple data ('SIMD') architecture is presented. Specifically, a method to unpack a fixed-width bit values in a bit stream to a fixed width byte stream in a SIMD architecture is presented. A method to unpack variable-length byte packed values in a byte stream in a SIMD architecture is presented. A method to decompress a run length encoded compressed bit-vector in a SIMD architecture is presented. A method to return the offset of each bit set to one in a bit-vector in a SIMD architecture is presented. A method to fetch bits from a bit-vector at specified offsets relative to a base in a SIMD architecture is presented. A method to compare values stored in two SIMD registers is presented.

Efficient hardware instructions for single instruction multiple data processors

Techniques are described for materializing pre-computed results of expressions. In an embodiment, a set of one or more column units are stored in volatile or non-volatile memory. Each column unit corresponds to a column that belongs to an on-disk table within a database managed by a database server instance and includes data items from the corresponding column. A set of one or more virtual column units, and data that associates the set of one or more column units with the set of one or more virtual column units, are also stored in memory. The set of one or more virtual column units includes a particular virtual column unit storing results that are derived by evaluating an expression on at least one column of the on-disk table.

Materializing expressions within in-memory virtual column units to accelerate analytic queries

Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data.

Mirroring, in memory, data from disk to improve query performance

To prioritize repopulation of in-memory compression units (IMCU), a database server compresses, into an IMCU, a plurality of data units from a database table. In response to changes to any of the plurality of data units within the database table, the database server performs the steps of: (a) invalidating corresponding data units in the IMCU; (b) incrementing an invalidity counter of the IMCU that reflects how many data units within the IMCU have been invalidated; (c) receiving a data request that targets one or more of the plurality of data units of the database table; (d) in response to receiving the data request, incrementing an access counter of the IMCU; and (e) determining a priority for repopulating the IMCU based, at least in part, on the invalidity counter and the access counter.

Prioritizing repopulation of in-memory compression units

Techniques are provided for more efficiently using the bandwidth of the I/O path between a CPU and volatile memory during the performance of database operation. Relational data from a relational table is stored in volatile memory as column vectors, where each column vector contains values for a particular column of the table. A binary-comparable format may be used to represent each value within a column vector, regardless of the data type associated with the column. The column vectors may be compressed and/or encoded while in volatile memory, and decompressed/decoded on-the-fly within the CPU. Alternatively, the CPU may be designed to perform operations directly on the compressed and/or encoded column vector data. In addition, techniques are described that enable the CPU to perform vector processing operations on the column vector values.

Growing community of inventors

San Carlos, CA, United States of America

Allison Holloway