Sap Se

Sybase, Inc.

Other

Rolando Blanco

Mihnea Andrei

Anil Kumar Goel

Colin Florendo

Martin Heidel

Chaitanya Gottipati

Thomas Legler

Muhammed Sharique

Michael Muehle

Carsten Thiel

Peter Steinemann

Ivan Schreter

Guenter Radestock

Vivek Kandiyanallur

Joern Schmidt

Bernhard Scheirle

Robert Schulze

Thorsten Glebe

Reza Sherkat

David Wein

Andreas Tonder

Shiping Chen

Sarika Iyer

Nirvik Basak

Surendra Vishnoi

David Edward DeHaan

Jörn Schmidt

Christian Bensberg

Anisoara Nica

Alexander Boehm

Thomas Peh

Sebastian Seifert

Christian Lemke

Adrian Dragusanu

Deepak Shrivastava

Stephan Kottler

Mark Delafranier

Günter Radestock

Liang Song

Suprio Ray

Mihnea André

Harshada Chavan

Awisha Makwana

Xun Chen

A method may include receiving a first transaction inserting a record into a database and a second transaction deleting the record from the database. A validity period for the record may be determined based on a first commit time at which the first transaction is committed and a second commit time at which the second transaction is committed. A current table and/or a history table of a system versioned table may be updated to include the record based on the validity period of the record. One or more temporal operations may be performed based on the system versioned table. For example, a time travel operation may be performed to retrieve, based on the system versioned table, one or more records that are valid at a given point in time. Related systems and computer program products are also provided.

System versioned table with transactional system time

Disclosed herein are system, method, and computer program product embodiments for converting column loadable data in a database into a page loadable format. When a column oriented in-memory database needs to perform an operation on a table, the affected columns must be loaded fully into memory. In some cases, a database may partition tables, allowing for column fragments corresponding to those partitions to be loaded into memory. These column fragments may exceed available memory, such that it is beneficial to convert the column fragment into a page loadable format. Approaches are discussed herein for performing the conversion without the need to update global metadata during the process, allowing for multiple parallel column fragments to be updated, allowing data manipulation language (DML) operations to continue on the column during the conversion process, and reducing the impact of blocked parallel savepoint operations during the conversion process by chunking the conversion process.

In-place load unit conversion

A computer implemented method for manipulating data comprises receiving log transaction information from a primary database system at a secondary database system and parsing the log transaction information to identify data objects that have been modified on the primary database system. The secondary database system determines whether the data objects are present in main memory on the secondary database system and whether the log transaction corresponds to a data manipulation language (DML) query or a data definition language (DDL) query. If the data objects are not in the main memory and the query is a DML query, they are copied from persistence data volumes to the main memory. The data objects in main memory are modified by replaying the log transactions on the data objects in the main memory. Then, the data objects in the persistence data volumes are modified. Related apparatus, systems, techniques and articles are also described.

Replay of redo log records in persistency or main memory of database systems

Persisting data in logical tiers of an in-memory database based on aging rules of the database system. The database system can be configured to age data by moving the data from a first logical tier supporting high performance data access to a second logical tier supporting lower performance data access, thereby improving the operational efficiency of the database system.

Providing database storage to facilitate the aging of database-accessible data

Objects across multiple database manipulation language (DML) redo log records are cached, during continuous replay of redo log records on a secondary system, for the same table partition to enable reuse of such cached objects. Later, these cached objects can be reused as they are accessed during the sequential processing of DML redo records for a specific table partition.

Caching DML statement context during asynchronous database system replication

A computer implemented method for initializing a secondary database system includes receiving table state information from a primary database system at a secondary database system. The table state information includes information identifying which tables had an open operation during a savepoint event. Metadata associated with the tables is parsed to identify table state information from the metadata and a lock is created for each table identified as having an open operation during the savepoint event. Afterwards, log transaction information is sequentially parsed. Related apparatus, systems, techniques and articles are also described.

Persistence and initialization of synchronization state for serialized data log replay in database systems

A computer implemented method for manipulating data comprises receiving log transaction information from a primary database system at a secondary database system and parsing the log transaction information to identify data objects that have been modified on the primary database system. The secondary database system determines whether the data objects are present in main memory on the secondary database system. If the data objects are not in the main memory, they are copied from persistence data volumes to the main memory. The data objects are modified by replaying the log transactions on the data objects in the main memory. Then, the data objects in the persistence data volumes are modified. Related apparatus, systems, techniques and articles are also described.

Common runtime and recovery framework for manipulating data in main memory of database systems

A system includes a volatile random access memory storing first header data and a first data block, the first header data including descriptive information and a first pointer to the first data block, and a non-volatile random access memory. The system includes determination of a memory size associated with the first header data and the first data block, allocation of a first memory block of the non-volatile random access memory based on the determined memory size, determination of an address of the non-volatile random access memory associated with the allocated first memory block, and writing of the descriptive information and a binary copy of the first data block at the address of the non-volatile random access memory.

Storage of data structures in non-volatile memory

Systems and methods for integrating the lifecycle of nonvolatile memory blocks with the transactional guarantees of a database are disclosed. One method includes creating a first fragment of a first database column in a volatile memory system, receiving a first pointer to a first block of a second non-volatile memory (NVM) system, the first block associated with the first fragment of the first database column, and populating the first block of the second non-volatile memory system using the first pointer. The method further includes committing the first block of the second non-volatile memory system and associating a first block identifier of the first block of the second non-volatile memory system with the first fragment of the first database column, while blocking creation of a database savepoint, and then unblocking creation of the database savepoint. A block key associated with a first fragment may comprise various identifiers such as block, column, table and partition identifiers. A first fragment may be a main column fragment and comprise a first dictionary such as sorted dictionary, while a second fragment may be a delta column fragment and comprise a second dictionary, which may be a non-sorted dictionary.

Lifecycle management for data in non-volatile memory including blocking creation of a database savepoint and associating non-volatile memory block identifiers with database column fragments

The subject matter disclosed herein provides methods for reformatting a page due to a rollover. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each page can have one or more rows for storing the column of data. The column of data in the in-memory array can be monitored for a change. A rollover can be performed on at least one of the pages based on the change. The rollover can reformat the at least one page by rewriting metadata associated with the at least one page. Related apparatus, systems, techniques, and articles are also described.

Optimized rollover processes to accommodate a change in value identifier bit size and related system reload processes

At least one read operation is concurrently performed with at least one write operation that each insert a key/value pair into a backing array of a backing hash table of a hash table forming part of a columnar in-memory database. The backing array maps a plurality of pointers each to a respective bucket. Each bucket includes at least one state bit and a hashed value of a corresponding key. Thereafter, for each write operation, a first available position in the backing array at which a pointer to a new bucket containing the key/value pair can be inserted is iteratively determined (such that each first available position has no corresponding pre-existing pointer). Subsequently, for each write operation, the pointer to the new bucket containing the key/value pair is inserted at the corresponding first determined position in the backing array. Related apparatus, systems, techniques and articles are also described.

Versioned insert only hash table for in-memory columnar stores

As part of a columnar in-memory database, value identifiers are inserted into a backing array in-memory until such time that it is determined that such backing array does not have sufficient capacity. A new backing array is then generated that includes the value identifiers in the old backing array and which has sufficient capacity. The old backing array can be flushed from memory when there are no active operations using such backing array. Such an arrangement allows for readers and non-structural writers to operate concurrently.

N-bit compressed versioned column data array for in-memory columnar stores

Disclosed herein are system, method, and computer program product embodiments for bloom filter versioning. A request to add a new value to a first bitmap is received, wherein the first bitmap includes a bit sequence corresponding to a most recently added value. It is determined that the first bitmap has exceeded a threshold. A second bitmap is generated, the second bitmap including more bits than the first bitmap. The first bitmap is deleted based upon a determination that a number of readers accessing the first bitmap is zero.

Versioned bloom filter

A delta store giving row-level versioning semantics to a non-row-level versioning underlying store is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the main store allows only non-concurrent transactions on a same table and the delta store has a plurality of row-visibility bitmaps implementing a row-level versioning mechanism that allows concurrent transactions on the same table. A local RID space is established for a table fragment, that for each table in the database, the data of the table is stored in one or more main table fragment in the main store and in one or more delta table fragments in the delta store. Each table fragment has a local RID space, and the local RID space is a collection of one-based contiguous integer local RIDs (Row IDs) describing local positions of the rows of the table fragment.

Multiple RID spaces in a delta-store-based database to support long running transactions

Growing community of inventors

Waterloo, Canada

Rolando Blanco