Oracle International Corporation

Sankar Subramanian

Andrew Witkowski

Huagang Li

Srikanth Bellamkonda

Nathan K Folkert

Abhinav Gupta

Rafi Ahmed

Tolga Bozkaya

Lei Sheng

Srikanth Bondalapati

Mohamed Ziauddin

Sabina Petride

Angela Amor

You Jung Kim

Venkatesh Sakamuri

Qingyuan Kong

Thierry Cruanes

Juan R Loaiza

Tirthankar Lahiri

Beda Christoph Hammerschmidt

Bhaskar Ghosh

Ajit Mylavarapu

Praveen T J Kumar

Vinita Subramanian

Janaki Latha Lahorani

Chun-Chieh Lin

Gerald Venzl

Disclosed herein are techniques for storing, within a database system, metadata that indicates an intended usage (IU). Once created, an IU may be assigned to a column to (a) indicate how the column is intended to be used, and (b) affect how the database server behaves when database operations involve values from the column. The IU assigned to a column supplements, but does not replace, the datatype definition for the column. Each IU may have an IU-bundle. The IU-bundle of an IU indicates how the database server behaves with respect to any column that is assigned the IU. For example, the IU-bundle may indicate constraints that the database server must validate during operations on values from columns assigned to the IU. Techniques are also described for implementing multi-column IUs and flexible IUs.

Extending database data with intended usage information

Techniques for the automatic creation and maintenance of zone maps are provided. In one technique, a set of data sets is identified. For each data set, a data set width is determined based on a maximum value in the data set and a minimum value in the data set. One or more zones within the data set are identified. For each zone, a zone width is determined based on a difference between a maximum value in that zone and a minimum value in that zone. An aggregate zone width is generated that is based on the zone width of each zone. Based on the data set width and the aggregate zone width, it is determined whether to automatically generate a zone map for the data set.

Automatic creation and maintenance of zone maps

Techniques are described for storing and maintaining, in a materialized view, bitmap data that represents a bitmap of each possible distinct value of an expression and rewriting a query for a count of distinct values of the expression using the materialized view. The materialized view contains bitmap data that represents a bitmap of each possible distinct value of a first expression, and aggregate values of additional expressions, and is stored in memory or on disk by a database system. The database system receives a query that requests a number of distinct values, of the first expression, and an aggregate value for an additional expression. In response, the database system, rewrites the query to: compute the number of distinct values by counting the bits in the bitmap data of the materialized view that are set to the first value, and obtains the aggregate value for the additional expression in the materialized view.

Bitmap-based count distinct query rewrite in a relational SQL algebra

Techniques are described for parallel processing of database queries with an inverse distribution function by a database management system (DBMS). To improve the execution time of a query with an inverse distribution function, the data set referenced in the inverse distribution function is range distributed among parallel processes that are spawned and managed by a query execution coordinator process (QC), in an embodiment. The parallel executing processes sort each range of the data set in parallel, while the QC determines the location(s) of inverse distribution function values based on the count of values in each range of the data set. The QC requests the parallel processes to produce to the next stage of parallel processes the values at the location(s) in the sorted ranges. The next stage of parallel processes computes the inverse distribution function based on the produced values. Techniques are also described for parallel executing of queries that may additionally include another inverse distribution function, one or more non-distinct aggregate functions and one or more distinct aggregate functions.

Parallel processing of queries with inverse distribution function

Techniques are described herein for processing queries comprising joins specifying a plurality of tables. The techniques involve partitioning the tables by assigning rows to partitions. One or more partition maps may be generated to indicate the partitions. Subsequent tables may be partitioned based on the generated partition maps. The partitions may be used to determine results for the joins.

N-way hash join

A method and system for processing database queries containing aggregate functions. The query may specify fewer groups than there are processes available to process the queries. Further, the queries may target a set of rows and specify a sort-by key and a group-by key. The method and system further includes determining that the queries specify application of the aggregate function to each of a plurality of groups that may correspond to a plurality of distinct values of the group-by key and determining that plurality of processes are available to process the queries. The method and system also includes determining the plurality of ranges of a composite key that may be formed by combining the group-by key and the sort-by key and assigning each range of the plurality ranges to a corresponding process to calculate the aggregate function.

Scalable multistage processing of queries with percentile functions

Techniques herein optimize sort-merge join method for a band join. In an embodiment, for a query comprising a query block specifying a join between a first table and a second table, a band join condition is detected between the first table and the second table. Once the band join condition in detected, an execution plan is generated and executed. The execution of the execution plan includes: for a first row of at least a subset of first sorted rows, scanning second rows from a set of second sorted rows, joining each of said second rows with said first row, and ceasing to scan when encountering a row from the second sorted rows that falls outside a bound of said band join condition. Techniques also include parallelizing a workload by overlapping the distribution of rows to the same slave process and computing cost and cardinality estimation for enhanced band join.

Sort-merge band join optimization

Attributes and semantics of duplicate insignificance that are inherent or inferred in a database language statement are detected. Also, a join operation that is inherent or inferred in the database language statement is detected and examined for join semantics. The join semantics specifies or refers to a driving table to be subjected to a hash join operation that may populate one or more hash buckets. The optimizer and the execution layers may use cost estimation or heuristics to assign the left and right table roles to the tables involved in the join. The hash join operation removes left table duplicates during population of the hash buckets, resulting in full or partial duplicate elimination that occurs during the hash join operation.

Duplicate reduction or elimination with hash join operations

Techniques are described for leveraging column dictionaries of tables for join, group-by and expression evaluation operations. In an embodiment, a table is stored in one or more data units, each data unit's metadata containing dictionaries for stored columns. Rather than storing unencoded column values, the data units may store columns as column vectors of dictionary-encoded values, in an embodiment. When performing a join operation, a matching of values may be performed on the build-side table using the unencoded, unencoded, values stored in the join-key dictionary(s) of the probe-side table, thus, significantly reducing the number of searching and matching operations. In an embodiment, a group-by operation may be executed by performing partial aggregations based on unique group-by key values as stored in the one or more group-by key dictionaries. For an expression evaluation, only a single evaluation may be performed for each unique combination of expression-key values in a data unit by leveraging the one or more expression-key dictionaries.

Leveraging columnar encoding for query operations

A method, apparatus, and stored instructions are provided for the removal of redundant GROUP BY and/or DISTINCT. Every table in the FROM clause of the query block must be a qualified table for the GROUP-BY clause or the DISTINCT keyword in the SELECT clause of the query block to be removed. A table Tx that satisfies at least one of the following two conditions is referred to as a qualified table: (1) Tx has a non-null unique column Tx.u that appears on the GROUP BY clause or the SELECT clause that contains a DISTINCT keyword and (2) There is a qualified table Ty and Ty has a filtering join with Tx.

Redundant group by and distinct removal

Techniques are provided herein for processing a query using in-memory cursor duration temporary tables. The techniques involve storing a part of the temporary table in memory of nodes in a database cluster. A part of the temporary table may be stored in disk segments of nodes in the database cluster. Writer threads running on a particular node writes data for the temporary table to the memory of the particular node. Excess data may be written to the disk segment of the particular node. Reader threads running on the particular node reads data for the temporary table from the memory of the particular node and the disk segment of the particular node.

In-memory cursor duration temp tables

Techniques for query optimization for group-by extensions and distinct aggregate functions are provided. A query has an extended group-by clause with an extended group-by operator and a first set of group-by columns. The query has one or more distinct aggregate functions and one or more non-distinct aggregate functions. An initial subquery is constructed that generates a partially aggregated initial temporary (PAIT) table when executed. The initial subquery includes a GROUP BY clause with a second set of group-by columns that includes the first set of group-by columns of the extended group-by clause of the query and one or more columns specified by the one or more distinct aggregate functions. One or more subqueries are computed that compute groupings indicated by the extended group-by operator based on the PAIT table generated by the initial subquery.

Query optimization for group-by extensions and distinct aggregate functions

According to one aspect of the invention, for a database statement that specifies evaluating reporting window functions, a computation-pushdown execution strategy may be used for the database statement. The computation-pushdown execution plan includes producer operators and consolidation operators. Each producer operator computes a respective partial aggregation for each reporting window function based on a subset of rows, and broadcasts the respective partial aggregation. Each consolidation operator fully aggregates all partial aggregations broadcasted from the producer operators. Alternatively, an extended-data-distribution-key execution plan may be used. Each producer operator sends rows based on hash keys to sort operators for computing partial aggregations for at least one reporting window function based on a subset of rows. Each consolidation operator receives and fully aggregates all partial aggregations broadcasted from the sort operators.

Scalable and adaptive evaluation of reporting window functions

Techniques are described for parallelizing qualifying window functions without reliance on partitioning criteria specified in the command that includes the qualifying window functions. Because the techniques do not rely on partitioning criteria specified in the command, the techniques may be used on commands in which no partitioning criteria is specified. In general, the techniques involve partitioning the work of a qualified window function into ranges, and sharing precomputed aggregate values between computational elements to calculate the value for the window functions.

Growing community of inventors

Cupertino, CA, United States of America

Sankar Subramanian