Sap Se

Sap Ag

Christian Lemke

Sebastian Seifert

Robert Schulze

Mihnea Andrei

Colin Florendo

Reza Sherkat

Norman May

Amit Pathak

Adrian Dragusanu

Junze Bao

Wei Zhou

Frederic Junker

Ivan Schreter

Christoph Weyerhaeuser

Franz Faerber

Tobias Mindnich

Guenter Radestock

Rolando Blanco

Frederik Transier

Muhammed Sharique

Carsten Thiel

Kai-Uwe Sattler

Bernhard Scheirle

Surendra Vishnoi

A database management system is described that can encode data to generate a plurality of data vectors. The database management system can perform the encoding by using a dictionary. The database management system can adaptively reorder the plurality of data vectors to prepare for compression of the plurality of data vectors. During a forward pass of the adaptive reordering, most frequent values of a data vector of the plurality of data vectors can be moved-up in the data vector. During a backward pass of the adaptive reordering, content within a rest range of a plurality of rest ranges can be rearranged within the plurality of data vectors according to frequencies of the content. The reordering according to frequency can further sort the rest range by value. Related apparatuses, systems, methods, techniques, computer programmable products, computer readable media, and articles are also described.

Adaptive compression optimization for effective pruning

Systems and methods include reception of an instruction to perform a consistency check on compressed column data. In response to the instruction, a compression algorithm applied to uncompressed column data to generate the compressed column data is determined, one or more consistency checks associated with the compression algorithm are determined, wherein a first one or more consistency checks associated with a first compression algorithm are different from a second one or more consistency checks associated with a second compression algorithm, the one or more consistency checks are executed on the compressed column data, and, if the one or more consistency checks are not satisfied, a notification is transmitted to a user.

Consistency checks for compressed data

Disclosed herein are system, method, and computer-program product embodiments for generating a paged and in-memory representation of a database object. An embodiment operates by maintaining in-memory and paged form primitives unique to the database object or a substructure thereof in a database such that the in-memory and paged form primitives are capable of providing the in-memory and paged representations of the database objects, respectively. Thereafter, a load configuration for the database object is determined. Based on the load configuration, the in-memory and/or paged representations of the database object are generated using the in-memory form primitive or the paged form primitive unique to the database object, respectively. Subsequently, the in-memory and/or paged representations of the database object are stored in the database.

Hybrid column store providing both paged and memory-resident configurations

A system includes application of respective compression types to first data associated with each of a plurality of columns to generate compressed column data, determination of a first compression ratio for each of the plurality of columns based on the compressed column data, storage of the determined first compression ratios, application, for each of the plurality of columns, of the determined compression type to second data associated with the column to generate second compressed column data, determination of a second compression ratio for each of the plurality of columns based on the second compressed column data, determination of a value for each column based on the stored first compression ratio and the second compression ratio determined for the column, determination of a representative value of the determined values, and determination, based on the representative value, whether to re-determine a compression type for each of the plurality of columns.

Compression determination for column store

A system in which a volatile random access memory stores first header data, second header data, a first logical array in a first contiguous memory block and a second logical array in a second contiguous memory block. Each array position of the first logical array stores a database column value, and each array position of the second logical array stores an indication of a number of consecutive occurrences of a database column value. The first header data includes a first pointer to the first memory block, and the second header data includes a second pointer to the second memory block. A memory size is determined associated with the first header data, the second header data, the first memory block, and the second memory block, a first memory block of the non-volatile random access memory is allocated based on the determined memory size, an address of the random access memory associated with the allocated first memory block is determined, and a portion of the first header data, a portion of the second header data, a binary copy of the first memory block and a binary copy of the second memory block are written at the address of the random access memory.

Storage of run-length encoded database column data in non-volatile memory

A system includes a volatile random access memory storing first header data and a first data block, the first header data including descriptive information and a first pointer to the first data block, and a non-volatile random access memory. The system includes determination of a memory size associated with the first header data and the first data block, allocation of a first memory block of the non-volatile random access memory based on the determined memory size, determination of an address of the non-volatile random access memory associated with the allocated first memory block, and writing of the descriptive information and a binary copy of the first data block at the address of the non-volatile random access memory.

Storage of data structures in non-volatile memory

A method can include: reordering an enriched inverted index associated with a database, the enriched inverted index including a first inverted list having a first plurality of current document identifiers of records that contain a first data value, the enriched inverted index further including a first data structure storing enrichment data, the reordering of the enriched inverted index comprising: generating an ordinal sequence corresponding to an order of a first plurality of current document identifiers that include a change of at least one of the first plurality of current document identifiers to a new document identifier; determining a reordered ordinal sequence corresponding to a sorted order of the second plurality of document identifiers; separately reordering, based at least on the reordered ordinal sequence, the first plurality of current document identifiers in the first inverted list and the enrichment data in the first data structure.

Reordering of enriched inverted indices

Disclosed herein are system and method embodiments for generating a paged inverted index. An embodiment is generated by storing a first data structure and the second data structure in a plurality of pages, where the plurality of pages are stored in the one or more memories. The first data structure is stored in the plurality of pages and includes a plurality of value identifiers, where a value identifier corresponds to an offset. The second data structure stored in the plurality of pages includes a plurality of row positions, wherein a row position is at a location that corresponds to the offset in the first data structure and identifies a position of row in a table that stores data associated with the value ID.

Paged inverted index

An additional data structure can be initialized for a column of compressed data to include a prefix storing, for each block of values in the column, a total number of bits set in previous blocks in the bit vector. A block number can be determined for a target block of the plurality of blocks, for example by checking whether or not a specified row number is located in the prefix. If the specified row number is located in the prefix, the prefix value of the prefix is returned, the most frequently occurring value is returned if a corresponding bit in the bit vector in the specified row number is not located in the prefix, or a position of the specified row in an index vector for the column is returned.

Growing community of inventors

Walldorf, Germany

Christian Lemke