Sap Se

Other

Till Merker

Paul Willems

Manuel Mayr

Christina Teflioudi

Andreas Amler

Wolfgang Stephan

Joerg Fellmann

Oliver Horn

Christian Bensberg

Jonathan Dees

Timm Steinbeck

Dirk Thomsen

Nico Bohnsack

Jin Su Lee

Arnaud Lacurie

Dan Bi Park

Dennis Felsing

Heesik Shin

Alexander Ostertag

Canberk Morelli

Cheng-Lun Li

Mortiz Thomas

Myunggu Kang

Florian Drescher

In some implementations, there is provided receiving a query request including a top k query operator for query plan generation, optimization, and execution; generating a query plan that includes at least one pipeline of a plurality of operators, wherein the at least one pipeline is associated with a directed acyclic graph; detecting in the query plan a first operator in the at least one pipeline that causes an early exit; and in response to the early exit by the first operator during query execution, processing back through the directed acyclic graph to identify at least one preceding operator that should or should not run given the early exit of the first operator.

Handling early exit in a pipelined query execution engine via backward propagation of early exit information

In an example embodiment, the execution of a command (such as JSON_TABLE) to convert language-independent data format data to a structured table is performed by parallel processing the data in parallel and without buffering all of the input and output. More specifically, a pipelined approach is used that is parallelizable and reduces memory requirements. Input rows are parallel processed within chunks, while produced rows are output directly.

Pipelined language-independent data format table

A method, a system, and a computer program product for executing a blocked index join. One or more join values for joining data stored in a database in response to a query for accessing data stored in the database are identified. The database stores data in a plurality of tables. Each table has a plurality of columns and a plurality of rows. A mapping of one or more rows in the plurality of rows corresponding to one or more join values to a number of rows including one or more join values is identified. Based on the mapping, a join of one or more join values based on the rows including one or more join values is executed. The joined values are outputted.

Blocked index join

A method for executing a query may include generating a partition value identifier for a partitioned table. The partitioned table may include a main fragment including a main dictionary storing a first value and a main value identifier corresponding to the first value and a delta fragment including a delta dictionary storing a second value and a delta value identifier corresponding to the second value. The partition value identifier may be set based at least in part on the first value and the second value. The generated partition value identifier and a corresponding one of the main value identifier and the delta value identified may be maintained as part of a mapping. A query to group data stored in the partitioned table may be received. The query may be executed by at least using the mapping.

Partition-local partition value identifier grouping

In an example embodiment, an early filter is applied with a query plan using intra-pipeline predicate back-propagation. Specifically, the query plan may be thought of as a pipeline of operations. A runtime variable var may be introduced, and a specialized filter using var may be pushed down below the join operation. Var is a dynamic variable that is updated to track a value from the sort or similar operation (such as max (heap), reflecting the maximum value of a max-heap used by the sort or similar operation). The runtime variable gets initialized once the heap reaches a minimum number of elements (such as K in the case of a top K sort). Thus, before the heap reaches that minimum number of elements, the filter does not apply. Once the heap does reach that minimum number of elements, the filter does apply and acts to filter elements. Since the filter has been pushed down below the join operation, this saves processing cycles.

Adaptive early filtering with intra-pipeline predicate back-propagation

A method for performing a multi-fragment index scan operation included in a query pipeline of a query accessing a database table may include performing an index scan on a first chunk of a first fragment of the database table and a second chunk of a second fragment of the database table. A first output of the index scan on the first chunk of the first fragment of the database table and a second output of the index scan on the second chunk of the second fragment of the database table may be merged to generate a partial result set for ingestion by a next operation in the query pipeline. The multi-fragment index scan operation may continue with index scans on subsequent chunks of the first fragment and second fragment of the database table. Related systems and computer program products are also provided.

Multi-fragment index scan

A system and method is provided for implementing a table scan predicate with integrated semi-join filter. The method includes receiving a query including a request to join first data from a first dimension table and second data from a second dimension table with fact data from a fact table. The method includes applying a first dynamic predicate to the first data by collecting the first data based on a first expression of the query and filtering the first column. The method also includes applying a second dynamic predicate to the second data by collecting the second data based on a second expression of the query and filtering the second column. The method also includes executing the query by at least scanning the fact table based on the query, the first filtered column, and the second filtered column.

Table scan predicate with integrated semi-join filter

In an example embodiment, a specialized execution process for a Disk Group-By operator with at least one distinct or ordered aggregation is provided. This specialized execution process involves writing all linked lists to disk (optionally this is performed only if the memory budget is coming close to being surpassed). Initially, a separate disk partition can be created per group, but once a partition becomes too large a new partition for the group is created. A partition is considered too large if it would not be able to be loaded into memory without surpassing the memory budget. As a result, every group can have a list of one or more disk partitions attached to it. The data belonging to each group is sorted using a sorted merge from disk.

Disk group-by with sorted merge

A database management system (DBMS) traverses an operator topology of a query plan to identify pipelines within the query plan. The query plan may include at least a first shared subplan reference, where the first shared subplan reference refers to a first shared subplan, and where the first shared subplan is associated with a shared node in the operator topology. Also, the DBMS assigns, based on an order of execution, pipeline indices to the pipelines in the query plan. Additionally, the DBMS tracks pipeline dependencies of the query plan. For each shared subplan reference, the database management system specifies the shared subplan reference as a pipeline breaker if its pipeline has a dependency to a pipeline with an index greater than a corresponding shared subplan. Otherwise, the shared subplan reference is specified as a non-pipeline breaker. Then, the query plan is updated based on these shared subplan reference specifications.

Shared subplan pipelining inference

A method may include receiving, from a first thread of a first operator performing a first operation of a query, a first request for memory allocation or memory deallocation. In response to the first request, a first local data structure of the first thread may be accessed to identify the first operator associated with the first request. A first memory usage metric for the first thread of the first operator may be updated based on the first request. A third memory usage metric for the first operator may be determined based on the first memory usage metric and a second memory usage metric for a second thread of the first operator. A fourth memory usage metric for a second operator may be determined based on a fifth memory usage metric for a third thread used by the second operator to perform a second operation of the query.

Telemetry for query processing memory usage

In some implementations, there is provided a method including receiving a query request including a join, wherein the join includes a range between a first predicate of the join and a second predicate of the join; generating a query plan including an index join operator; executing the query plan including the index join operator including getting, from the sorted dictionary, the first value identifier, the second value identifier, and the one or more intervening value identifiers between the first value identifier and the second value identifier and executing the index join operator using the first value identifier, the second value identifier, and the one or more intervening value identifiers to obtain a result set.

Index join query optimization

Systems, methods, and articles of manufacture, including computer program products, provide a system including at least one data processor and at least one memory storing instructions which, when executed by the at least one data processor, cause operations comprising: generating, by a database execution engine, a query plan including a plurality of operators; inserting, by the database execution engine, an enforce compilation operator into the query plan that includes the plurality of operators, the plurality of operators comprising a first operator, the enforce compilation operator, and a second operator; executing at least the first operator of the query plan; in response to executing the first operator, evaluating, by the database execution engine, an output of the first operator to determine whether a condition is satisfied; and in response to the condition being satisfied, triggering, by the database execution engine, a just-in-time compilation of the second operator.

JIT code compilation during runtime based on runtime statistics

Methods and systems for object-based text searching using group score expressions are provided. A method may include receiving a query including a request to search specified columns of a table for a set of search terms, and a group score filter for use in filtering the table based at least on a group score associated with a plurality of groups of rows of the table, determining the group score for each of a plurality of groups of rows of the table, filtering the table based at least on the group score filter included in the query and the group score determined for each of the plurality of groups of rows of the table, and providing at least one group of rows of the plurality of groups of rows that includes at least the portion of the set of search terms.

Object-based text searching using group score expressions

Systems and methods are provided for text searching using partial score expressions. A method may include receiving a query to search for a search term in at least a first column of a first table and a second column of a second table, scanning the first column and the second column for at least a portion of the search term, generating a first partial score table, generating a second partial score table, determining a combined score for each row in the first column and/or the second column containing at least the portion of the search term based at least on a join of the first partial score table and the second partial score table, and providing, in response to the query and based at least on the combined score, a row of the first column and/or the second column including at least the portion of the search term.

Text searching using partial score expressions

A system for generating a mixed query plan including operator alternatives. A method may include generating a mixed query plan including a first operator selected as a pre-compiled operator; generating the mixed query plan including a second operator selected as operator alternatives, the operator alternatives configuring the second operator as pre-compiled or code-generating alternatives; delaying selection of one of the operator alternatives until additional information regarding the mixed query plan becomes available; generating the mixed query plan including a third operator selected as a code-generating operator; and selecting, given the third operator representing the additional information, one of the operator alternatives to enable execution of the mixed query plan using the selected operator alternative. Related methods and articles of manufacture are also described.

Growing community of inventors

Sandhausen, Germany

Till Merker