Sap Se

Sap Ag

Kai Stammerjohann

Frederik Transier

Christian Bensberg

Peter Sanders

Nico Bohnsack

Christian Mathis

Stephan Kottler

Martin Richtarsky

Ingo Müller

Various embodiments for a disk-based merge for hash maps are described herein. An embodiment operates by identifying a plurality of hash maps with a plurality of disjunctions. The hash values of each of the entries may be moved to memory and compared for a particular disjunction. A data value with a lower hash value as determined based on the comparison is selected and stored in a merged hash map. The process is repeated until all the data values have been compared. A query is received, and processed based on the merged hash map.

Disk-based merge for hash maps

Embodiments are described for a database management system comprising a memory and at least one processor coupled to the memory. The at least one processor is configured to receive a query that corresponds to a data slice and determine a bloom filter based on the query. The at least one processor is further configured to determine that the data slice includes data requested by the query based on the bloom filter and in response to determining that the data slice includes the data requested by the query, load the data slice to the memory.

Bloom filter for avoiding partition loading

Various embodiments for a data compression store and search system are described herein. An embodiment operates by receiving a query to search a plurality of documents. It is determined that the plurality of documents are stored in a compressed format comprising a plurality of shells and a plurality of snippets. A first snippet that includes the query identifier is identified, and a first shell including a reference to the first snippet is identified. A readable version of a first document of the plurality of documents is assembled from the first shell. The assembled readable version of the first document is provided responsive to the query.

Data compression, store, and search system

Disclosed herein are system, method, and computer program product embodiments for providing data aggregation based on hash map data structures. An embodiment operates by receiving a query specifying an input table and an aggregate function. The embodiment then generates a first thread-local hash map and a second thread-local hash map for the input table and performs a probing function associated with the first thread-local hash map. The embodiment then determines, based on the first thread-local hash map, an index cardinality associated with the input table. The embodiment then, in response to the determination that the index cardinality exceeds the threshold, performs a duplicate function associated with the second thread-local hash map and generates a second thread-local copy map. The embodiment then merges the first thread-local hash map and the second thread-local copy map, thereby generating a merged hash map.

Data aggregation based on hash map data structures

Various embodiments for a disk-based merge for combining merged hash maps are described herein. An embodiment operates by identifying a first hash map and a second hash map, and comparing a first hash value from the first hash map with a second hash value from the second hash map, with the lowest index values. A lowest hash value is identified based on the comparison, and an entry corresponding to the lowest hash value is stored in a combined hash map. This process is repeated until all of the hash values from both the first set of hash values and the second set of hash values are stored in the combined hash map. A query is received, and processed based on the combined hash map.

Disk-based merge for combining merged hash maps

Various embodiments for a disk-based merge for hash maps are described herein. An embodiment operates by identifying a plurality of hash maps with a plurality of disjunctions, ordering the one or more entries in each disjunction based on the hash value, and assigning an index value to each data value based on the ordering. The hash values of each of the entries may be moved to memory and compared for a particular disjunction. A data value with a lower hash value as determined based on the comparison is selected and stored in a merged hash map. The process is repeated until all the data values have been compared. A query is received, and processed based on the merged hash map.

A system includes reception of a query specifying a database table, an aggregate function, a plurality of grouping columns and an aggregation column, determination of whether a tuple of a first row of the grouping columns in the database tables is unique in the grouping columns, storage, if it is determined that the tuple of the first row of the grouping columns is unique in the grouping columns, of a result set entry including the tuple and an aggregation value determined based on a value of the aggregation column in the first row, and, if it is determined that the tuple of the first row of the grouping columns is not unique in the grouping columns, determination of a key based on the tuple and performance of a hash lookup using the key.

Aggregation over high-cardinality dimensions using uniqueness indices

A method, a system and a computer program product for execution of queries. A first query is executed using a first query execution plan to generate a first result data by applying a first query operator to a first data stored in a first storage location. The first result data is provided to a second query execution plan for execution of a second query. The second query is executed on a second data stored in a second storage location in accordance with the second query execution plan. The second query execution plan includes a second query operator. Based on the execution of the second query, a second result data is generated in accordance with the second query execution plan by at least applying the second query operator to at least the provided first result data.

Query plan sharing

In one respect, there is provided a method. The method can include processing a first data chunk to generate a first intermediate result. A key map can be generated based on a determination that a quantity of the key-value pairs in the first intermediate result exceeds a threshold. The key map can be generated to include keys in the first intermediate result. A second data chunk can be processed to generate a second intermediate result. The second data chunk can be processed based on the key map. The processing of the second data chunk can include omitting a key-value pair in the second data chunk from being inserted into the second intermediate result based on a key associated with the key-value pair being absent from the key map. A preview of the processing of the dataset can be generated based on the first intermediate result and the second intermediate result.

Preview data aggregation

A new dictionary can be created for a result column in a query plan operation executed on a database. The result column can be generated by multiple worker jobs running in parallel to read tasks from a shared queue as part of a query plan operation that includes a group-by column within an input set of input columns. The group-by column can include an original dictionary for all values contained within the group-by column If the new dictionary has fewer entries than the original dictionary for the group-by column such that mapping is required between old value identifiers within the group-by column and new value identifiers within the result column, the old value identifiers are renamed to the new value identifiers using a mapping vector.

Lock-free generation of columns with minimal dictionaries after parallel aggregation

According to some embodiments, a system and method for a parallel join of relational data tables may be provided by calculating, by a plurality of concurrently executing execution threads, hash values for join columns of a first input table and a second input table; storing the calculated hash values in a set of disjoint thread-local hash maps for each of the first input table and the second input table; merging the set of thread-local hash maps of the first input table, by a second plurality of execution threads operating concurrently, to produce a set of merged hash maps; comparing each entry of the merged hash maps to each entry of the set of thread-local hash maps for the second input table to determine whether there is a match, according to a join type; and generating an output table including matches as determined by the comparing.

Hash-join in parallel computation environments

A system, method and medium may provide determination of a first plurality of a plurality of data records assigned to a first processing unit, identification of a first record of the first plurality of data records, the first record associated with a first key value, generation of a first dictionary entry of a first dictionary for the first key value, storage of a first identifier of the first record as a tail identifier and as a head identifier in the first dictionary entry, storage an end flag in a first shared memory location, the first shared memory location associated with the first record, identification of a second record of the first plurality of data records, the second record associated with the first key value, replacement of the tail identifier in the first dictionary entry with a second identifier of the second record, and storage of the first identifier in a second shared memory location, the second shared memory location associated with the second record.

Growing community of inventors

Wiesloch, Germany

Kai Stammerjohann