Sap Se

Other

Till Merker

Paul Willems

Manuel Mayr

Christina Teflioudi

Andreas Amler

Wolfgang Stephan

Joerg Fellmann

Oliver Horn

Christian Bensberg

Jonathan Dees

Timm Steinbeck

Dirk Thomsen

Nico Bohnsack

Jin Su Lee

Arnaud Lacurie

Dan Bi Park

Heesik Shin

Dennis Felsing

Canberk Morelli

Cheng-Lun Li

Alexander Ostertag

Myunggu Kang

Florian Drescher

Mortiz Thomas

A method for executing a query may include generating a partition value identifier for a partitioned table. The partitioned table may include a main fragment including a main dictionary storing a first value and a main value identifier corresponding to the first value and a delta fragment including a delta dictionary storing a second value and a delta value identifier corresponding to the second value. The partition value identifier may be set based at least in part on the first value and the second value. The generated partition value identifier and a corresponding one of the main value identifier and the delta value identified may be maintained as part of a mapping. A query to group data stored in the partitioned table may be received. The query may be executed by at least using the mapping.

Partition-local partition value identifier grouping

In an example embodiment, an early filter is applied with a query plan using intra-pipeline predicate back-propagation. Specifically, the query plan may be thought of as a pipeline of operations. A runtime variable var may be introduced, and a specialized filter using var may be pushed down below the join operation. Var is a dynamic variable that is updated to track a value from the sort or similar operation (such as max (heap), reflecting the maximum value of a max-heap used by the sort or similar operation). The runtime variable gets initialized once the heap reaches a minimum number of elements (such as K in the case of a top K sort). Thus, before the heap reaches that minimum number of elements, the filter does not apply. Once the heap does reach that minimum number of elements, the filter does apply and acts to filter elements. Since the filter has been pushed down below the join operation, this saves processing cycles.

Adaptive early filtering with intra-pipeline predicate back-propagation

A method for performing a multi-fragment index scan operation included in a query pipeline of a query accessing a database table may include performing an index scan on a first chunk of a first fragment of the database table and a second chunk of a second fragment of the database table. A first output of the index scan on the first chunk of the first fragment of the database table and a second output of the index scan on the second chunk of the second fragment of the database table may be merged to generate a partial result set for ingestion by a next operation in the query pipeline. The multi-fragment index scan operation may continue with index scans on subsequent chunks of the first fragment and second fragment of the database table. Related systems and computer program products are also provided.

Multi-fragment index scan

A system and method is provided for implementing a table scan predicate with integrated semi-join filter. The method includes receiving a query including a request to join first data from a first dimension table and second data from a second dimension table with fact data from a fact table. The method includes applying a first dynamic predicate to the first data by collecting the first data based on a first expression of the query and filtering the first column. The method also includes applying a second dynamic predicate to the second data by collecting the second data based on a second expression of the query and filtering the second column. The method also includes executing the query by at least scanning the fact table based on the query, the first filtered column, and the second filtered column.

Table scan predicate with integrated semi-join filter

In an example embodiment, a specialized execution process for a Disk Group-By operator with at least one distinct or ordered aggregation is provided. This specialized execution process involves writing all linked lists to disk (optionally this is performed only if the memory budget is coming close to being surpassed). Initially, a separate disk partition can be created per group, but once a partition becomes too large a new partition for the group is created. A partition is considered too large if it would not be able to be loaded into memory without surpassing the memory budget. As a result, every group can have a list of one or more disk partitions attached to it. The data belonging to each group is sorted using a sorted merge from disk.

Disk group-by with sorted merge

A database management system (DBMS) traverses an operator topology of a query plan to identify pipelines within the query plan. The query plan may include at least a first shared subplan reference, where the first shared subplan reference refers to a first shared subplan, and where the first shared subplan is associated with a shared node in the operator topology. Also, the DBMS assigns, based on an order of execution, pipeline indices to the pipelines in the query plan. Additionally, the DBMS tracks pipeline dependencies of the query plan. For each shared subplan reference, the database management system specifies the shared subplan reference as a pipeline breaker if its pipeline has a dependency to a pipeline with an index greater than a corresponding shared subplan. Otherwise, the shared subplan reference is specified as a non-pipeline breaker. Then, the query plan is updated based on these shared subplan reference specifications.

Shared subplan pipelining inference

A method may include receiving, from a first thread of a first operator performing a first operation of a query, a first request for memory allocation or memory deallocation. In response to the first request, a first local data structure of the first thread may be accessed to identify the first operator associated with the first request. A first memory usage metric for the first thread of the first operator may be updated based on the first request. A third memory usage metric for the first operator may be determined based on the first memory usage metric and a second memory usage metric for a second thread of the first operator. A fourth memory usage metric for a second operator may be determined based on a fifth memory usage metric for a third thread used by the second operator to perform a second operation of the query.

Telemetry for query processing memory usage

In some implementations, there is provided a method including receiving a query request including a join, wherein the join includes a range between a first predicate of the join and a second predicate of the join; generating a query plan including an index join operator; executing the query plan including the index join operator including getting, from the sorted dictionary, the first value identifier, the second value identifier, and the one or more intervening value identifiers between the first value identifier and the second value identifier and executing the index join operator using the first value identifier, the second value identifier, and the one or more intervening value identifiers to obtain a result set.

Index join query optimization

Systems, methods, and articles of manufacture, including computer program products, provide a system including at least one data processor and at least one memory storing instructions which, when executed by the at least one data processor, cause operations comprising: generating, by a database execution engine, a query plan including a plurality of operators; inserting, by the database execution engine, an enforce compilation operator into the query plan that includes the plurality of operators, the plurality of operators comprising a first operator, the enforce compilation operator, and a second operator; executing at least the first operator of the query plan; in response to executing the first operator, evaluating, by the database execution engine, an output of the first operator to determine whether a condition is satisfied; and in response to the condition being satisfied, triggering, by the database execution engine, a just-in-time compilation of the second operator.

JIT code compilation during runtime based on runtime statistics

Methods and systems for object-based text searching using group score expressions are provided. A method may include receiving a query including a request to search specified columns of a table for a set of search terms, and a group score filter for use in filtering the table based at least on a group score associated with a plurality of groups of rows of the table, determining the group score for each of a plurality of groups of rows of the table, filtering the table based at least on the group score filter included in the query and the group score determined for each of the plurality of groups of rows of the table, and providing at least one group of rows of the plurality of groups of rows that includes at least the portion of the set of search terms.

Object-based text searching using group score expressions

Systems and methods are provided for text searching using partial score expressions. A method may include receiving a query to search for a search term in at least a first column of a first table and a second column of a second table, scanning the first column and the second column for at least a portion of the search term, generating a first partial score table, generating a second partial score table, determining a combined score for each row in the first column and/or the second column containing at least the portion of the search term based at least on a join of the first partial score table and the second partial score table, and providing, in response to the query and based at least on the combined score, a row of the first column and/or the second column including at least the portion of the search term.

Text searching using partial score expressions

A system for generating a mixed query plan including operator alternatives. A method may include generating a mixed query plan including a first operator selected as a pre-compiled operator; generating the mixed query plan including a second operator selected as operator alternatives, the operator alternatives configuring the second operator as pre-compiled or code-generating alternatives; delaying selection of one of the operator alternatives until additional information regarding the mixed query plan becomes available; generating the mixed query plan including a third operator selected as a code-generating operator; and selecting, given the third operator representing the additional information, one of the operator alternatives to enable execution of the mixed query plan using the selected operator alternative. Related methods and articles of manufacture are also described.

Automatic selection of precompiled or code-generated operator variants

In some implementations, there is provided a method that includes receiving a query request including a top k query operator for query plan generation, optimization, and execution, wherein k defines a threshold limit of query results for the top k query operator; inserting into a query plan a check operator associated with the top k query operator; in response to executing the query plan, checking, by the check operator, whether an early exit occurs due to the top k query operator reaching the threshold limit; in response to the early exit occurring due to the top k query operator reaching the threshold limit, stopping processing, by the check operator, including opening of another fragment of a database table; and in response to the early exit not occurring, allowing, by the check operator, the opening of the other fragment of the database table.

Custom table scan for top k queries

Described herein are systems and method for providing data transfer in a computer-implemented database from a database extension layer. A data server associated with a database receives a request to transfer data stored in a database extension layer of the database. Input data chunks are collected from the database extension layer until a configured row count limit is reached. Row positions are determined from the input data chunks. Value identifiers corresponding to the row positions are determined. Values corresponding to the value identifiers are retrieved. Output data is generated based on the values corresponding to the value identifiers.

Data transfer in a computer-implemented database from a database extension layer

In some example embodiments, there is provided a method that includes receiving a query for query plan generation and query plan optimization; detecting whether the query plan includes a restart pipeline, the restart pipeline including one or more first operators that cause a query plan restart; detecting whether the query plan includes a result pipeline, the result pipeline including one or more second operators that provide result streaming; determining whether there is a dependency between execution of the restart pipeline and execution of the result pipeline; in response to determining there is no dependency, reordering the restart pipeline in the query plan such that the restart pipeline will be executed before the result pipeline in order to enable result streaming optimization; and providing the reordered query plan for execution. Related systems and computer program products are also provided.

Growing community of inventors

Sandhausen, Germany

Till Merker