Sap Se

International Business Machines Corporation

Other

Manuel Mayr

Till Merker

Paul Willems

Wolfgang Stephan

Paul Edward McKenney

Oliver Petrik

Nico Bohnsack

Andreas Amler

Cheng-Lun Li

Katharina Probst

Olaf Rutz

Maurice Debatin

Jochen Röhrig

In an example embodiment, the execution of a command (such as JSON_TABLE) to convert language-independent data format data to a structured table is performed by parallel processing the data in parallel and without buffering all of the input and output. More specifically, a pipelined approach is used that is parallelizable and reduces memory requirements. Input rows are parallel processed within chunks, while produced rows are output directly.

Pipelined language-independent data format table

Arrangements for a reading scheme for column-oriented databases are provided. A virtual column may be generated by applying a structured query language operation on distinct values in a column of data in a table. Based on applying the structured query language operation, the virtual column may be populated with corresponding precalculated values. The virtual column may be compressed with dictionary-based compression. The compressing may include generating a column of value identifiers, each of the value identifiers representing a distinct value in the column of data. A database query against the compressed virtual column may be received. A result of the database query may be returned by retrieving one or more of the precalculated values from the compressed virtual column.

Calculated dictionary column readers

A system and method is provided for implementing a table scan predicate with integrated semi-join filter. The method includes receiving a query including a request to join first data from a first dimension table and second data from a second dimension table with fact data from a fact table. The method includes applying a first dynamic predicate to the first data by collecting the first data based on a first expression of the query and filtering the first column. The method also includes applying a second dynamic predicate to the second data by collecting the second data based on a second expression of the query and filtering the second column. The method also includes executing the query by at least scanning the fact table based on the query, the first filtered column, and the second filtered column.

Table scan predicate with integrated semi-join filter

A database management system (DBMS) traverses an operator topology of a query plan to identify pipelines within the query plan. The query plan may include at least a first shared subplan reference, where the first shared subplan reference refers to a first shared subplan, and where the first shared subplan is associated with a shared node in the operator topology. Also, the DBMS assigns, based on an order of execution, pipeline indices to the pipelines in the query plan. Additionally, the DBMS tracks pipeline dependencies of the query plan. For each shared subplan reference, the database management system specifies the shared subplan reference as a pipeline breaker if its pipeline has a dependency to a pipeline with an index greater than a corresponding shared subplan. Otherwise, the shared subplan reference is specified as a non-pipeline breaker. Then, the query plan is updated based on these shared subplan reference specifications.

Shared subplan pipelining inference

In some implementations, there is provided a method including receiving a query request including a join, wherein the join includes a range between a first predicate of the join and a second predicate of the join; generating a query plan including an index join operator; executing the query plan including the index join operator including getting, from the sorted dictionary, the first value identifier, the second value identifier, and the one or more intervening value identifiers between the first value identifier and the second value identifier and executing the index join operator using the first value identifier, the second value identifier, and the one or more intervening value identifiers to obtain a result set.

Index join query optimization

Systems, methods, and articles of manufacture, including computer program products, provide a system including at least one data processor and at least one memory storing instructions which, when executed by the at least one data processor, cause operations comprising: generating, by a database execution engine, a query plan including a plurality of operators; inserting, by the database execution engine, an enforce compilation operator into the query plan that includes the plurality of operators, the plurality of operators comprising a first operator, the enforce compilation operator, and a second operator; executing at least the first operator of the query plan; in response to executing the first operator, evaluating, by the database execution engine, an output of the first operator to determine whether a condition is satisfied; and in response to the condition being satisfied, triggering, by the database execution engine, a just-in-time compilation of the second operator.

JIT code compilation during runtime based on runtime statistics

A method for on-demand access to table partitions may include receiving, at a data execution engine, a query accessing a database table divided into a first partition and a second partition stored at different nodes of the database. The data execution engine may respond to the query by acquiring a lock on the database table and instantiating data structures to support the execution of the query without loading any of the partitions of the database table into a main memory. Instead, the loading of the individual partitions of the database table may be delegated to the operators tasked with performing the operations included in the execution plan for the query. Each operator may refrain from loading those partitions of the database table that are determined to be unnecessary for performing the individual operations. Related systems and computer program products are also provided.

On-demand access of database table partitions

A table scan predicate with integrated semi-join filter is provided. A method includes receiving a query including: a request to join first data from a first table and second data from a second table, a first predicate for use in a table scan of the second table, and a second predicate including an expression associated with the first data from the first table and a reference to a column associated with the second data from the second table. The method may include transforming the second predicate into a dynamic predicate for execution of the query. the method may include applying the dynamic predicate to at least the first data. The method may include executing the query by at least scanning the second table based on the first predicate and filtered first data from the application of the dynamic predicate. Related systems and articles of manufacture are provided.

A query is received at a database execution engine. A query plan is generated by the database execution engine and for execution of the query. The query plan includes a first operator including computer executable instructions that upon execution queries a data structure of a database, determines whether the data structure satisfies a precondition for execution of a second operator of the query plan, and generates a signal indicating that the precondition is not satisfied. Execution of the query plan is initiated. A new query plan is generated by the database execution engine and in response to receiving the signal indicating that the precondition is not satisfied. Related systems, methods, and articles of manufacture are also described.

Generation of query execution plans

A query is received at a database execution engine. A query plan including a sub plan structured as a directed acyclic graph is determined by the database execution engine. A set of trees characterizing the sub plan is generated by the database execution engine and using the directed acyclic graph. The set of trees include a first tree and a second tree, the first tree including at least one leaf characterizing a memory store operation and the second tree including a root characterizing a memory access operation. The set of trees are stored for use in execution of the query at run time. Related systems, methods, and articles of manufacture are also described.

Transforming directed acyclic graph shaped sub plans to enable late materialization

A method for enriching map functions of a digital geographical map is provided. Additional data are stored in a storage unit outside map data of the digital geographical map and direct related map metadata. Elements of the additional data are correlated to coordinates of the digital geographical map. Further, access is provided to the additional data via an authentication service while navigating the geographical map.

Map enriched by data other than metadata

An optimized skiplist iterator technique for RCU-protected skiplists. A skiplist iterator operation may attempt to validate a cached pointer hint that references a skiplist element of an RCU-protected skiplist, the skiplist element having an associated first key. If the pointer hint is validated, the pointer hint may be dereferenced to access the skiplist element. A pointer in the skiplist element may be dereferenced to advance to a next skiplist element of the RCU-protected skiplist, the next skiplist element having an associated second key that is larger than the first key. If the pointer hint is not validated, a value-based skiplist iterator operation may be performed that includes traversing the RCU-protected skiplist using the first key to find the next skiplist element having the second key. A new pointer hint that references the next skiplist element may be cached for use in a next invocation of the skiplist iterator operation.

Growing community of inventors

Walldorf, Germany

Manuel Mayr