Sap Se

Sap Ag

Frederik Transier

Kai Stammerjohann

Christian Bensberg

Nico Bohnsack

Franz Faerber

Christian Mathis

Steffen Geissinger

Wolfgang Stephan

Thomas Peh

Sebastian Seifert

Holger Schwedes

Christian Lemke

Peter Sanders

Joern Schmidt

Martin Richtarsky

Ingo Müller

Embodiments are described for a database management system comprising a memory and at least one processor coupled to the memory. The at least one processor is configured to receive a query that corresponds to a data slice and determine a bloom filter based on the query. The at least one processor is further configured to determine that the data slice includes data requested by the query based on the bloom filter and in response to determining that the data slice includes the data requested by the query, load the data slice to the memory.

Bloom filter for avoiding partition loading

Various embodiments for a data compression store and search system are described herein. An embodiment operates by receiving a query to search a plurality of documents. It is determined that the plurality of documents are stored in a compressed format comprising a plurality of shells and a plurality of snippets. A first snippet that includes the query identifier is identified, and a first shell including a reference to the first snippet is identified. A readable version of a first document of the plurality of documents is assembled from the first shell. The assembled readable version of the first document is provided responsive to the query.

Data compression, store, and search system

Disclosed herein are system, method, and computer program product embodiments for providing data aggregation based on hash map data structures. An embodiment operates by receiving a query specifying an input table and an aggregate function. The embodiment then generates a first thread-local hash map and a second thread-local hash map for the input table and performs a probing function associated with the first thread-local hash map. The embodiment then determines, based on the first thread-local hash map, an index cardinality associated with the input table. The embodiment then, in response to the determination that the index cardinality exceeds the threshold, performs a duplicate function associated with the second thread-local hash map and generates a second thread-local copy map. The embodiment then merges the first thread-local hash map and the second thread-local copy map, thereby generating a merged hash map.

Data aggregation based on hash map data structures

The present disclosure involves systems, software, and computer implemented methods for compression operation combining duplicate index entries independent of a data model. One example method includes operations to identify an update to at least one entry in a compressed index that includes a plurality of entries, each associated with a unique entry ID. An entry ID of each entry associated with the update are identified. A self-update is performed for each entry not associated with the entry IDs associated with the update, which comprises inserting a value associated with those non-updated entries to an uncompressed index in connection with that entry's corresponding entry ID. For each entry associated with the update, a particular update value from the identified update is inserted into the uncompressed index associated with the particular entry ID. When completed, the uncompressed index is compressed into a new version of the compressed index.

Index compression using reordering and self-updates

Various embodiments for a disk-based merge for combining merged hash maps are described herein. An embodiment operates by identifying a first hash map and a second hash map, and comparing a first hash value from the first hash map with a second hash value from the second hash map, with the lowest index values. A lowest hash value is identified based on the comparison, and an entry corresponding to the lowest hash value is stored in a combined hash map. This process is repeated until all of the hash values from both the first set of hash values and the second set of hash values are stored in the combined hash map. A query is received, and processed based on the combined hash map.

Disk-based merge for combining merged hash maps

Various embodiments for a disk-based merge for hash maps are described herein. An embodiment operates by identifying a plurality of hash maps with a plurality of disjunctions, ordering the one or more entries in each disjunction based on the hash value, and assigning an index value to each data value based on the ordering. The hash values of each of the entries may be moved to memory and compared for a particular disjunction. A data value with a lower hash value as determined based on the comparison is selected and stored in a merged hash map. The process is repeated until all the data values have been compared. A query is received, and processed based on the merged hash map.

Disk-based merge for hash maps

A system includes reception of a query specifying a database table, an aggregate function, a plurality of grouping columns and an aggregation column, determination of whether a tuple of a first row of the grouping columns in the database tables is unique in the grouping columns, storage, if it is determined that the tuple of the first row of the grouping columns is unique in the grouping columns, of a result set entry including the tuple and an aggregation value determined based on a value of the aggregation column in the first row, and, if it is determined that the tuple of the first row of the grouping columns is not unique in the grouping columns, determination of a key based on the tuple and performance of a hash lookup using the key.

Aggregation over high-cardinality dimensions using uniqueness indices

In one aspect, there is provided a method. The method may include adding a tuple phrase to a phrase index as part of a lazy creation of the phrase index. The adding may include adding the tuple phrase to a dictionary mapped to the phrase index, adding to the phrase index a document identifier and a positional information for the tuple phrase, and adapting a first threshold and a second threshold value, the first threshold representing a threshold lower estimate of query processing burden associated with a least burdensome tuple phrase in the dictionary and the second threshold representing a top estimate of query processing burden associated with a most burdensome tuple phrase in the dictionary. The method may further include sending a query response including a generated query result corresponding to the tuple phrase.

Self-adapting resource aware phrase indexes

A method, a system and a computer program product for execution of queries. A first query is executed using a first query execution plan to generate a first result data by applying a first query operator to a first data stored in a first storage location. The first result data is provided to a second query execution plan for execution of a second query. The second query is executed on a second data stored in a second storage location in accordance with the second query execution plan. The second query execution plan includes a second query operator. Based on the execution of the second query, a second result data is generated in accordance with the second query execution plan by at least applying the second query operator to at least the provided first result data.

Query plan sharing

In one respect, there is provided a method. The method can include processing a first data chunk to generate a first intermediate result. A key map can be generated based on a determination that a quantity of the key-value pairs in the first intermediate result exceeds a threshold. The key map can be generated to include keys in the first intermediate result. A second data chunk can be processed to generate a second intermediate result. The second data chunk can be processed based on the key map. The processing of the second data chunk can include omitting a key-value pair in the second data chunk from being inserted into the second intermediate result based on a key associated with the key-value pair being absent from the key map. A preview of the processing of the dataset can be generated based on the first intermediate result and the second intermediate result.

Preview data aggregation

A method can include: reordering an enriched inverted index associated with a database, the enriched inverted index including a first inverted list having a first plurality of current document identifiers of records that contain a first data value, the enriched inverted index further including a first data structure storing enrichment data, the reordering of the enriched inverted index comprising: generating an ordinal sequence corresponding to an order of a first plurality of current document identifiers that include a change of at least one of the first plurality of current document identifiers to a new document identifier; determining a reordered ordinal sequence corresponding to a sorted order of the second plurality of document identifiers; separately reordering, based at least on the reordered ordinal sequence, the first plurality of current document identifiers in the first inverted list and the enrichment data in the first data structure.

Reordering of enriched inverted indices

Flexible text searching for data objects can be implemented. For example, inexact matching of key name terms can be supported. An inverted index can be used to find data objects having key names with terms that are a superset of the key name term being searched. A similar index can be used for values, and the search results can be determined via an intersection. Pattern searching and other features can be supported. Text searching for schema-less collections of data objects can be supported for use in enterprise computing scenarios.

Flexible text searching for data objects of object notation

According to some embodiments, a system and method for a parallel join of relational data tables may be provided by calculating, by a plurality of concurrently executing execution threads, hash values for join columns of a first input table and a second input table; storing the calculated hash values in a set of disjoint thread-local hash maps for each of the first input table and the second input table; merging the set of thread-local hash maps of the first input table, by a second plurality of execution threads operating concurrently, to produce a set of merged hash maps; comparing each entry of the merged hash maps to each entry of the set of thread-local hash maps for the second input table to determine whether there is a match, according to a join type; and generating an output table including matches as determined by the comparing.

Hash-join in parallel computation environments

A system, method and medium may provide determination of a first plurality of a plurality of data records assigned to a first processing unit, identification of a first record of the first plurality of data records, the first record associated with a first key value, generation of a first dictionary entry of a first dictionary for the first key value, storage of a first identifier of the first record as a tail identifier and as a head identifier in the first dictionary entry, storage an end flag in a first shared memory location, the first shared memory location associated with the first record, identification of a second record of the first plurality of data records, the second record associated with the first key value, replacement of the tail identifier in the first dictionary entry with a second identifier of the second record, and storage of the first identifier in a second shared memory location, the second shared memory location associated with the second record.

Growing community of inventors

Heidelberg, Germany

Frederik Transier