Sap Se

Chaitanya Gottipati

Colin Florendo

Michael Muehle

Rolando Blanco

Nirvik Basak

Mihnea Andrei

Martin Heidel

Peter Steinemann

Ivan Schreter

Thomas Legler

Thorsten Glebe

Reza Sherkat

Vivek Kandiyanallur

Sarika Iyer

Souvik Pal

Jörn Schmidt

Norman May

Thomas Peh

Muhammed Sharique

Shiping Chen

Carsten Thiel

Prasanta Ghosh

Deepak Shrivastava

Stephan Kottler

Awisha Makwana

Amit Saonerkar

Günter Radestock

Disclosed herein are system, method, and computer program product embodiments for storing a large object (LOB) in a database. An embodiment operates by identifying the LOB to be stored in an on-disk store of the database and determining the size of the LOB. Thereafter, an index vector for the on-disk store to include an identifier corresponding to the LOB is created. Additionally, a dictionary for the on-disk store to include a copy of the identifier and corresponding LOB data is also created. The LOB data corresponds to the LOB itself or a LOB location inside of the on-disk store based on the LOB's size.

Paged hybrid LOBs

Disclosed herein are system, method, and computer program product embodiments for converting column loadable data in a database into a page loadable format. When a column oriented in-memory database needs to perform an operation on a table, the affected columns must be loaded fully into memory. In some cases, a database may partition tables, allowing for column fragments corresponding to those partitions to be loaded into memory. These column fragments may exceed available memory, such that it is beneficial to convert the column fragment into a page loadable format. Approaches are discussed herein for performing the conversion without the need to update global metadata during the process, allowing for multiple parallel column fragments to be updated, allowing data manipulation language (DML) operations to continue on the column during the conversion process, and reducing the impact of blocked parallel savepoint operations during the conversion process by chunking the conversion process.

In-place load unit conversion

System, method, and various embodiments for providing a metadata based memory management system are described herein. An embodiment operates by determining metadata of a table of a database indicating whether data of the table is to be stored in memory or disk storage, wherein the table comprises a plurality of columns and a plurality of rows. A first column of the table is identified. A hierarchy of the metadata for the table including a column value, partition value, and table value is determined, wherein the column value is prioritized over the partition value, and wherein the partition value is prioritized over the table value. The data of the table stored in the column is loaded from the disk storage into the memory based on the hierarchy. A query is processed based on the data loaded into the memory.

Hierarchical metadata enhancements for a memory management system

System, method, and various embodiments for providing a metadata converter and memory management system are described herein. An embodiment operates by determining that first metadata corresponding to a table of a database comprising load preferences for a column level for a plurality of columns of the table, wherein the load preferences include either column load or page load. It is determined that the database is enabled with both load preferences for a table level and load preferences partition level, in addition to load preferences for the column level. Values for the load preferences are automatically assigned for both the table level and the partition level in second metadata, and wherein the second metadata preserves the load preferences for the column level of the first metadata. A query against the table based on load preferences from the second metadata.

Metadata converter and memory management system

A computer implemented method for manipulating data comprises receiving log transaction information from a primary database system at a secondary database system and parsing the log transaction information to identify data objects that have been modified on the primary database system. The secondary database system determines whether the data objects are present in main memory on the secondary database system and whether the log transaction corresponds to a data manipulation language (DML) query or a data definition language (DDL) query. If the data objects are not in the main memory and the query is a DML query, they are copied from persistence data volumes to the main memory. The data objects in main memory are modified by replaying the log transactions on the data objects in the main memory. Then, the data objects in the persistence data volumes are modified. Related apparatus, systems, techniques and articles are also described.

Replay of redo log records in persistency or main memory of database systems

Systems, apparatus, techniques, articles, and methods include a unified table container for storing data and metadata in pages organized in one or more page chains. A determination is made as to whether the query includes a data definition statement. In response to determining that the query includes a data definition statement, in-memory runtime data is persisted to a persistent volume that is configured to be accessed by the database system. The in-memory runtime data includes one or more first data fields and one or more second data fields that are different from the one or more first data fields. Furthermore, the unified table container includes a persistent runtime data descriptor that is associated with the one or more first data fields and persists the first one or more first data fields when persisting the in-memory runtime data.

Runtime data persistency for in-memory database systems

Objects across multiple database manipulation language (DML) redo log records are cached, during continuous replay of redo log records on a secondary system, for the same table partition to enable reuse of such cached objects. Later, these cached objects can be reused as they are accessed during the sequential processing of DML redo records for a specific table partition.

Caching DML statement context during asynchronous database system replication

A computer implemented method for updating a secondary database includes receiving transaction log information from a primary database system at a secondary database system. The transaction log information comprises a first marker identifying a beginning of an operation comprising a plurality of sub-operations and a second marker identifying an end of the operation. The transaction log information is sequentially parsed, a first marker is identified, and a lock state is created after replaying the first marker to ensure proper serialization of the plurality of sub-operations. The lock state is terminated after replaying the second marker. Related apparatus, systems, techniques and articles are also described.

Synchronization mechanism for serialized data log replay in database systems

A computer implemented method for initializing a secondary database system includes receiving table state information from a primary database system at a secondary database system. The table state information includes information identifying which tables had an open operation during a savepoint event. Metadata associated with the tables is parsed to identify table state information from the metadata and a lock is created for each table identified as having an open operation during the savepoint event. Afterwards, log transaction information is sequentially parsed. Related apparatus, systems, techniques and articles are also described.

Persistence and initialization of synchronization state for serialized data log replay in database systems

As part of a columnar in-memory database, value identifiers are inserted into a backing array in-memory until such time that it is determined that such backing array does not have sufficient capacity. A new backing array is then generated that includes the value identifiers in the old backing array and which has sufficient capacity. The old backing array can be flushed from memory when there are no active operations using such backing array. Such an arrangement allows for readers and non-structural writers to operate concurrently.

N-bit compressed versioned column data array for in-memory columnar stores

Disclosed herein are system, method, and computer program product embodiments for inverted indexing. An embodiment operates by receiving a request to add a new document identifier (doc ID) to an inverted index. A memory block corresponding to where the doc ID is to be added is determined or located, and the available space remaining in the memory block is determined. If the memory block has available space, the doc ID is added to the memory block.

Inverted indexing

Disclosed herein are system, method, and computer program product embodiments for bloom filter versioning. A request to add a new value to a first bitmap is received, wherein the first bitmap includes a bit sequence corresponding to a most recently added value. It is determined that the first bitmap has exceeded a threshold. A second bitmap is generated, the second bitmap including more bits than the first bitmap. The first bitmap is deleted based upon a determination that a number of readers accessing the first bitmap is zero.

Growing community of inventors

Pune, India

Chaitanya Gottipati