Sap Se

Sybase, Inc.

Other

David Wein

Ivan Schreter

Mihnea Andrei

Panfeng Zhou

Dirk Thomsen

Colin Florendo

Anil Kumar Goel

Thomas Legler

Steffen Geissinger

Thorsten Glebe

Rolando Blanco

Shiping Chen

Guenter Radestock

Michael Muehle

Martin Heidel

Mark Delafranier

Frank Benkstein

Steffen Geiβinger

The subject matter disclosed herein relates to out of memory error handling in a database system. A database operation can be received by an in-memory database. The database operation can be for a database object stored in a database table that can be represented as a plurality of pages that can be persisted in a page chain. The in-memory database can reserve out of memory space sufficient to load a predetermined number of pages in the page chain into memory. The in-memory database can iteratively process each page in the page chain until completion of the database operation. The iterative process can include loading the page from persistence into memory of the in-memory database, performing at least a portion of the database operation using the loaded page, and unloading the page from the memory of the in-memory database. Related apparatus, systems, techniques, and articles are also described.

Out of memory error handling for data definition language operations

Disclosed herein are system, method, and computer program product embodiments for altering the data type of a column in a database. An embodiment operates by converting an original dictionary associated with a column into a new dictionary. The new dictionary stores the values of the original dictionary using a different data type. An index vector containing the keys of the original dictionary is then updated to contain the associated keys of the new dictionary. Because the size of the original dictionary is often substantially smaller than the number of rows in the associated column, this dictionary conversion decreases the computation cost to the database system of altering the data type of the column and reduces or even minimizes database downtime for users.

Altering data type of a column in a database

The subject matter disclosed herein provides methods for the forced ordering of a dictionary based on row identifier values. A plurality of concurrent transactions can be handled in an in-memory database. At least one of the transactions can include at least one write operation to a dictionary. Each write operation can be assigned a row identifier (ID). At least one of the write operations can be written to the dictionary out of sequence. The sequence can be based on the row ID. Each row ID in the dictionary can be mapped to a corresponding value identifier in the dictionary. The dictionary can position the value identifiers so that the corresponding row IDs are in a sorted sequential order based on the row ID. Related apparatus, systems, techniques, and articles are also described.

Forced ordering of a dictionary storing row identifier values

Changes to data in a column in an in memory array resulting from operations of a transaction thread can be written to a database object associated with the column. The database object can be marked for materialization upon completion of all of the plurality of operations of the transaction thread such that the column and the changes are materialized from the in-memory array to one or more pages stored on a persistent storage based on the marking.

Full and partial materialization of data from an in-memory array to an on-disk page structure

The subject matter disclosed herein provides methods for reformatting a page due to a rollover. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each page can have one or more rows for storing the column of data. The column of data in the in-memory array can be monitored for a change. A rollover can be performed on at least one of the pages based on the change. The rollover can reformat the at least one page by rewriting metadata associated with the at least one page. Related apparatus, systems, techniques, and articles are also described.

Optimized rollover processes to accommodate a change in value identifier bit size and related system reload processes

Loading of table metadata into memory of an in-memory database is initiated. The table metadata is persisted across pages in a page chain. Thereafter, a plurality of metadata objects are materialized into memory that each include an object handle pinning an underlying persisted page in the page chain. The objects are populated with data from the underlying persisted pages. Subsequently, for one to many object relationships, a vector of object handles is generated that comprises a plurality of transient handles that each point to a different instance of a respective transient object. Alternatively, for one to one object relationships or many to one object relationships, an object handle to point to a respective linked object is generated. Related apparatus, systems, techniques and articles are also described.

Transient and persistent representation of a unified table metadata graph

The subject matter disclosed herein provides methods for determining the page on which a desired row position value is located. A table having a plurality of rows can be accessed. The rows can be distributed across one or more pages in an in-memory database. Each row can be associated with a unique row position value. Each page can be associated with a capacity that is representative of an amount of data stored on the page. A capacity index having a plurality of entries can be created to record changes in capacity between pages. Neighboring entries in the capacity index can have a different capacity. Each entry in the capacity index can correspond to a page. A page directory can be created based on the capacity index. The page directory can indicate all possible row position values associated with each page. Related apparatus, systems, techniques, and articles are also described.

Fast row to page lookup of data table using capacity index

The subject matter disclosed herein provides methods for the dual storage of data using an in-memory array and an on-disk page structure. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each of the one or more pages can have one or more rows for storing the column of data. Random access can be provided to a subset of the one or more rows by at least loading the subset of rows from the one or more pages to the in-memory array without loading all of the rows from the one or more pages. Related apparatus, systems, techniques, and articles are also described.

Dual data storage using an in-memory array and an on-disk page structure

The subject matter disclosed herein provides methods for materializing data from an in-memory array to one or more pages. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each of the one or more pages can have one or more rows for storing the column of data. At least one of the one or more pages can be marked for materialization. The column of data can be materialized by copying the data from the in-memory array to the one or more rows of the one or more pages. The materializing can be based on the marking. Related apparatus, systems, techniques, and articles are also described.

Materializing data from an in-memory array to an on-disk page structure

Disclosed herein are system, method, and computer program product embodiments for directly restoring a database from a log volume. An embodiment operates by reading one or more database transaction log records from a log volume. The embodiment then loads from a database table persistent storage one or more pages containing rows in the database table that will be read from or written to by the one or more database transaction log records. The embodiment then loads a dictionary for each column associated with the one or more database transaction log records. The embodiment then applies the one or more database transaction log records directly into the one or more loaded pages and the one or more dictionaries. Each dictionary and page modified in memory is then written to the database table persistent storage when a savepoint operation is performed.

Applying a database transaction log record directly to a database table container

Log records are accessed as part of a database operation in a database. The log records log insert, update, and delete operations in the database and include, for each row, a row position, a fragment identifier (ID), and a row ID. Thereafter, as part of the database operation, rows specified by the log records are located by: using the fragment identifier and the row position within the corresponding record of the log if the fragment with the corresponding fragment identifier is still available, otherwise, using the row identifier within the corresponding record of the log to look up the row position in an index of a corresponding row identifier column. The database operation is then finalized using the located rows. Related apparatus, systems, techniques and articles are also described.

Efficient database undo / redo logging

A set-oriented locking based on in-memory bitmap for a column store database is described. An example method includes establishing a column-based in-memory database including a main store and a delta store, where the delta store has a plurality of row-visibility lock bitmaps visible to transactions at various points in time. The lock bitmaps represent a bit encoding to indicate whether there are granted row locks tables in the database. A delete or an update statement is executed with a transaction on a table. A set of row locks on rows of the table manipulated by the delete or the update statement are requested to preclude other transactions from currently deleting or updating the same rows. Accordingly, set operations are performed on the lock bitmap to manage the set of row locks associated with the transaction.

Set-oriented locking based on in-memory bitmaps for a column-oriented database

A system and methodology providing workload management in a database cluster. In one embodiment, for example, a method for allocating workload amongst a plurality of database servers sharing access to data comprises steps of: periodically collecting information about workload at each database server; distributing the workload information to each of the plurality of database servers; in response to a client connected to a first database server requesting that work be performed, examining the workload information to determine whether the requested work should be performed at a second database server having a lower workload than the first database server; if it is determined that the work should be performed at the second database server, automatically transferring the client connection to the second database server and performing the requested work at the second database server; and otherwise, performing the requested work at the first database server.

Growing community of inventors

Saint Paul, MN, United States of America

David Wein