Oracle International Corporation

Rafi Ahmed

Mohamed Zait

Andrew Witkowski

Allison Waingold Lee

Srikanth Bellamkonda

Bhaskar Ghosh

Sankar Subramanian

Randall G Bello

Thierry Cruanes

Dinesh Das

Hong Su

Hermann Baer

Angela Amor

Mohamed Ziauddin

Hakan Jakobsson

Lei Sheng

Nirav Vyas

Allison Waingold

Srikanth Bondalapati

Chun-Chieh Lin

Gautam Kumar Parai

Sridhar Sarnobat

A new type of table join operation, outer semi join (OSJ), is provided, which can be used by an optimizer layer and an execution layer of a database management system (DBMS). OSJ combines the semantics of both left outer-join and semi-join. The concept of an anti-join marker (AJM) is also introduced, which specifies whether a matching row was not found between joined tables for each result row in an OSJ operation. The OSJ operation supports unnesting of a class of disjunctive ANY, ALL, EXISTS, NOT EXISTS, IN, and NOT IN subqueries for execution plan optimization. The disjunction may contain filter predicates. For unnesting, OSJ avoids the need of using a distinct operator on the right table and also supports using inequality (e.g. >, >=, <, <=) in connecting or correlating conditions of subqueries, rather than being limited to equality only.

Outer semi join for disjunctive subquery unnesting

In one technique, a definition of a materialized view is identified. Based on the definition, multiple candidate partitioning schemes are identified. A query is generated that indicates one or more of the candidate partitioning schemes. The query is then executed, where executing the query results in one or more partition counts, each corresponding to a different candidate partitioning scheme of the one or more candidate partitioning schemes. Based on the one or more partition counts, a candidate partitioning scheme is selected from among the plurality of candidate partitioning schemes. The materialized view is automatically partitioned based on the candidate partitioning scheme.

Automatic partitioning of materialized views

Techniques are described herein for subquery removal given two set operation-based subqueries in a query, where one subquery contains the result of the other. The described optimization technique of subquery removal is enabled by join and set operation-based containment of the set operation-based subqueries where semantic equivalence can be established for a given pair of set operation-based subqueries when some table(s)&#x2014;with associated join condition(s), correlation condition(s), and/or filter predicate(s)&#x2014;in one subquery are not considered. Subquery removal reduces multiple access to the same table and multiple evaluations of the same join conditions required to evaluate the query. When a subquery is removed from a disjunction, this may lead to other optimizations such as subquery unnesting, e.g., when the original query configuration would not permit query unnesting and the rewritten query (with one or more removed subqueries) permits unnesting.

Join-based containment for set operation-based sub query removal

Definitions of material views are automatically generated. In general, Automated MV generation identifies a set of candidates MVs by examining a working set of query blocks. Once the candidates are formed, the candidate MVs are further evaluated to calculate a benefit to the candidate MVs. An improved approach for generating a candidate set of MVs is described herein. The improved approach is referred to as the extended covering subexpression technique (ECSE). Under ECSE, various relationships between join sets other than strict equivalence are used to generate new resultant join sets. Such relationships include subset, intersection, superset, and union, which shall be described in further detail below. In some cases, relationships among resultant join sets and initial join sets are considered to generate new resultant join sets. The final resultant join sets are then used to form a candidate set of MVs.

Automatic generation of materialized views

A one-sided outer-join-based query is rewritten using a materialized view (MV), the definition of which includes the join but does not include a filter predicate from the query. The rewritten query nullifies data from the include-matching table, included in the MV, that does not satisfy the filter predicate. To improve accuracy of the query results, certain rows are removed from intermediate results of the query. To facilitate revising the query results for accuracy, the MV includes a unique column from the include-all table and also an indicator column that indicates whether a given row of the MV is an inner-join-type row or an anti-join-type row. The rewritten query adjusts the indicator values, in the indicator column, of MV rows that do not satisfy the filter to reflect a modified anti-join-type indicator value. Based on the modified indicator values and unique columns from include-all tables, accuracy of query results is attained.

Materialized view rewrite technique for one-sided outer-join queries

A method, apparatus, and stored instructions are provided for transforming a query representation by unnesting a predicate condition that is based on whether or not a result exists for a subquery of the predicate condition. An initial query representation is received. The initial query representation represents an initial query that includes an EXISTS-equivalent predicate or a NOT-EXISTS-equivalent predicate and at least one other predicate in a disjunction. The initial query representation is transformed into a semantically equivalent transformed query representation that represents a transformed query. The transformed query includes, instead of the EXISTS-equivalent predicate or a NOT-EXISTS-equivalent predicate, a join operator that references the data object. The transformed query representation, when used for execution, causes the at least one other predicate to be applied separately from a join operation caused by the join operator such that execution of the initial representation is semantically equivalent to execution of the transformed representation.

Optimization technique of generalized disjunctive semi/anti join

Techniques herein optimize sort-merge join method for a band join. In an embodiment, for a query comprising a query block specifying a join between a first table and a second table, a band join condition is detected between the first table and the second table. Once the band join condition in detected, an execution plan is generated and executed. The execution of the execution plan includes: for a first row of at least a subset of first sorted rows, scanning second rows from a set of second sorted rows, joining each of said second rows with said first row, and ceasing to scan when encountering a row from the second sorted rows that falls outside a bound of said band join condition. Techniques also include parallelizing a workload by overlapping the distribution of rows to the same slave process and computing cost and cardinality estimation for enhanced band join.

Sort-merge band join optimization

Attributes and semantics of duplicate insignificance that are inherent or inferred in a database language statement are detected. Also, a join operation that is inherent or inferred in the database language statement is detected and examined for join semantics. The join semantics specifies or refers to a driving table to be subjected to a hash join operation that may populate one or more hash buckets. The optimizer and the execution layers may use cost estimation or heuristics to assign the left and right table roles to the tables involved in the join. The hash join operation removes left table duplicates during population of the hash buckets, resulting in full or partial duplicate elimination that occurs during the hash join operation.

Duplicate reduction or elimination with hash join operations

A method, apparatus, and stored instructions are provided for the removal of redundant GROUP BY and/or DISTINCT. Every table in the FROM clause of the query block must be a qualified table for the GROUP-BY clause or the DISTINCT keyword in the SELECT clause of the query block to be removed. A table Tx that satisfies at least one of the following two conditions is referred to as a qualified table: (1) Tx has a non-null unique column Tx.u that appears on the GROUP BY clause or the SELECT clause that contains a DISTINCT keyword and (2) There is a qualified table Ty and Ty has a filtering join with Tx.

Redundant group by and distinct removal

A method, apparatus, and stored instructions are provided for transforming an initial query by eliminating a subquery nested within the initial query. The initial query may include an outer query that references a first instance of a particular data object. The initial query may also include set operator(s). At least a particular set operator may include a particular subquery that references a different instance of the particular data object. A query processor may transform the initial query to a transformed query that excludes the particular set operator and the particular subquery. The transformed query may instead include an added predicate that is based at least in part on the particular subquery. The added predicate may reference the first instance of the particular data object without referencing the different instance of the particular data object. The transformed query may be used for query execution instead of the initial query.

Transforming a query by eliminating a subquery

A method and apparatus for outer-joined and/or cross-joined table elimination for duplicate-insignificant queries is provided. A query block of a query specifies a join between a first table and a second table. The join is one of an outer join and a cross join. A first determination is made that the query block contains no references to the second table to be processed after the join. A second determination is made that the query block is duplicate-insignificant. In response to making the first determination and the second determination, the query block is transformed into a transformed query, where the second table is eliminated from a corresponding query block in the transformed query.

Outer-joined and cross-joined table elimination for duplicate-insignificant queries

Methods, stored instructions, and computer systems are described for selecting an OR-expansion state of a query. A query processor accesses a certain query and a control parameter. The certain query comprises disjunctive predicates, and the control parameter indicates a certain procedure of a plurality of alternative stored procedures for expanding queries comprising disjunctive predicates into semantically equivalent candidate queries comprising combinations of subqueries. Based at least in part on the control parameter, the query processor selects the certain procedure to expand the certain query into one or more certain semantically equivalent candidate queries, each comprising a different combination of two or more subqueries. The query processor causes cost-based evaluation of subject queries, including the certain query and the one or more certain semantically equivalent candidate queries. The query processor then selects a query of the subject queries for execution or for further evaluation or transformation.

Selecting from OR-expansion states of a query

Methods, computing devices, and computer-readable media are provided for transforming an initial query with a single-table outer join predicate to a second, semantically equivalent query that includes a multi-table predicate instead of the single-table outer join predicate. Because the second query does not include the single-table outer join predicate, the second query may be available for further transformations or optimization techniques, may produce a more efficient or otherwise different execution plan, or may be executed more efficiently or differently. The single-table outer join predicate may be in a full outer join ON clause of the initial query or nested in a left outer-joined LATERAL VIEW of the initial query. The multi-table predicate may be in a WHERE clause of the second query. Unlike the initial query, the second query allows use of a temporary organization of data from multiple tables for evaluation of multiple rows of the left outer join.

Transforming a single-table join predicate into a pseudo-join predicate

Methods, computer-readable media, and machines are provided for transforming a query by removing a correlated predicate from a lateral view in an initial query. A query processor may transform the lateral view into a regular view such that a transformed query with the regular view is semantically equivalent to the initial query. The transformed query may support a larger set of access paths, join methods, and join permutations, may qualify for other transformations or optimizations, or may otherwise be executed or optimized differently from the initial query. The lateral view may reference an aggregation function, and the query processor may add a GROUP BY operator to the transformed query to preserve, after the correlated predicate has been pulled out of the lateral view, an interaction that was present between the aggregation function and the correlated predicate when the correlated predicate was in the lateral view.

Technique of decorrelation of lateral views

A method, apparatus, and stored instructions are provided for transforming an initial query having a scalar subquery with correlated predicates into a transformed query having transformed predicates that are not in the scalar subquery. The transformed predicates correspond to the correlated predicates, but the transformed predicates are no longer correlated. The transformed predicates reference an outer join operation to preserve null values that may have been generated by the correlated predicates that were in the scalar subquery of the initial query. The transformed query is semantically equivalent to the initial query. When creating an execution plan for the transformed query, the query processor may choose from a much larger set of access paths, join methods, and join permutations than those that were available for the initial query. The transformed query may qualify for other transformations or optimizations, or may otherwise be executed or optimized differently from the initial query.

Growing community of inventors

Fremont, CA, United States of America

Rafi Ahmed