International Business Machines Corporation

Dennis Butterstein

Felix O Beier

Sabine Perathoner-Tschaffler

Einar Lueck

Daniel Martin

Martin Oberhofer

Namik Hrle

Terence Patrick Purcell

Oliver Benke

Thomas Schwarz

Tobias Ulrich Bergmann

Vassil Radkov Dimov

Tobias Kevin Fränzel

Kevin Pelletier

Huiyan Roy

Alexander Merschel

Nehal Dudani

Antje Dengler

Adnan Ahmad

Brigitte Gößler

Jéssica Costa Da Rocha

Samy Tafasca

Zeyuan Zhang

Aspects of the present disclosure relate to recovery of a target database system. The method comprises receiving multiple batches of database transactions to form the target database system. The method further comprises generating a single recovery bookmark before beginning processing of a current batch, wherein the single recover bookmark comprises: an earliest open entry for a previous batch, a latest commit entry for the previous batch, an earliest open entry for the current batch, a latest commit entry for the current batch, a transactional identifier, and list of tables to be processed; performing the sequence of database transactions; replacing the list of tables from the single recovery bookmark with a flag value upon completion; detecting a fault condition during the performance of the sequence of database transactions; recovering the target database system using the single recovery bookmark; and resuming the sequence of database transactions.

Recovery of a target database system

A method, computer system, and a computer program product for storing a graph is provided. The present invention may include providing a key-value store. The present invention may include storing a graph in the key-value store, wherein storing a specific node of the graph in the key-value store comprises: storing, as a key, identifiers of the specific node and of k-hop neighborhood of the specific node, and as value of the key, attribute values of the specific node, wherein the k-hop neighborhood of the specific node comprises a set of neighboring nodes that are reachable from the specific node within k edges.

Storing graph data in a key-value store

The present disclosure relates to a computer implemented method for data synchronization in a data analysis system. The data analysis system comprises a source and target database system. The method comprises: receiving a change record describing an operation performed on a data record in the source database system. The change record may be read for determining a value of a distribution key of the data record. The value of the distribution key may be used for selecting a target database node of the target database system where the operation is to be performed. A direct connection may be established to the selected target database node and the change record may be provided to the selected target database node through the direct connection.

Data distribution in data analysis systems

In an approach, a primary data center is provided including primary source and primary target database systems, where a function is activated causing the primary target database system to: include a copy of data and receive analysis queries from the primary source database system; and execute the analysis queries on data. A processor, in response to detecting a failure in the primary source database system: offloads queries intended for the primary source database system to a secondary source database system of a secondary data center also including a secondary target database system and a copy of data, where the function is deactivated. A processor, responsive to the primary target database system being available: receives analysis queries, processed by the secondary source database system, of the offloaded queries; and copies data to the secondary target database system. A processor causes the function to be activated in the secondary data center.

Operating a data center

A computer-implemented method for a crash recovery for linked databases may be provided. The linked databases comprise a source and related target database. Selected queries of the source database are transferred to the target database. The method comprises synchronizing selected portions of the source database with tables of an in-memory portion of target database and, storing persistently applied changes to the in-memory target database portion asynchronously and persistently. Upon a database crash of the target database system, the method also comprises restoring, the in-memory target database portion with the latest snapshot available, and applying, changes from the source database recovery log file that have a later timestamp than the latest snapshot available in the persistent target database storage of the in-memory target database portion.

Asynchronous persistency of replicated data changes in a database accelerator

A computer implemented method comprises receiving at least one log record. A type of operation performed in a source database system may be determined based on the log record. Based on the type of operation, an update strategy may be selected from a set of pre-defined update strategies, wherein the update strategy specifies a sequence of replication operations to be performed in order to apply changes described in the at least one log record to a target database system.

Data synchronization in a data analysis system

In an approach to improve online database replication by automating the upgrading of a database replications system online. Additionally, embodiments of the present invention stop an upgrade using a first incremental update strategy on data of a source database, identify an earliest open transaction from a first database to a second database, and identify a last committed log record identifier. Further, embodiments of the present invention execute an adaptive apply strategy on transactions including the earliest open transaction until the last committed log record identifier is reached by the adaptive apply strategy, and resume, by upgrade controller, the upgrade with a second incremental update strategy.

Automated online upgrade of database replication

A computer-implemented method for a crash recovery for linked databases may be provided. The linked databases comprise a source database and a related target database, and selected portions of content of tables of the source database are synchronized with respective portions of the content of the tables of the target database. Thereby, selected queries against a database management system comprising the source database are transferred to a database management system comprising the target database for processing. And the method also comprises synchronizing the portions of the content of the selected tables in the target database with the portions of the content of the selected tables in the source database by reading entries of a recovery log file relating to the source database and applying the read entries to the target database.

Volatile database caching in a database accelerator

The present disclosure relates to a method for data replication in a data analysis system (). A source database system () of the data analysis system () comprises a transaction log () storing log records generated by database transactions. The method comprises in response to determining () that a received log record is generated by a database transaction that rolls back a change of another database transaction whose log records are buffered in at least one record buffer, data indicative of a log record generated by the other database transaction buffering () in the compensation buffer tag data. The tag data may be used () for replicating to a target database system of the data analysis system buffered log records of the record buffer which are not marked as compensation records.

Data replication in a data analysis system

The present disclosure relates to a computer implemented method for applying changes into tables of a target database system using a data synchronization system, the data synchronization system being configured to automatically use at least one configurable parameter for applying a requested change in the target database system. The method provides a trained machine learning model, the machine learning model being configured to adjust the at least one configurable parameter based on a workload level. The method determines a current workload level at the target database system and uses the machine learning model for adjusting the at least one configurable parameter according to the determined workload level.

Machine learning based application of changes in a target database system

A method comprises receiving a stream of change log records from a source database system; generating change statistics based on a number of pending changes per table partition according to the change log records; estimating, based on performance statistics, a first amount of time for applying the pending changes to a target database system using an incremental update path; estimating, based on the performance statistics, a second amount of time for applying the pending changes to the target database using a bulk update path; dynamically selecting, based on comparison of the first amount of time with the second amount of time, one of the incremental update path and the bulk update path for applying the pending changes to the target database system; and applying the pending changes to the target database system using the selected update path.

Dynamic selection of synchronization update path

Approaches presented herein enable replicating data records between a source database and a target database. More specifically, for a batch of change records in a table received from the source database, a first estimated replication duration needed to apply the batch as a bulk change to the target is determined. For the same batch, a second estimated replication duration needed to apply a set of changes in a single row of the table to the target is determined based on time penalties for each column in the row. A threshold quantity of rows at which the first duration equals a summed total of second durations for the quantity is calculated. The bulk change is selected if a number of rows in the batch exceeds the threshold. Applying change records singly is selected if the number of rows in the batch is less than the threshold.

Dynamic selection of data apply strategy during database replication

Methods and systems for generating database statistics. Table statistics in a metadata catalog of a source database system are observed, statistics generation costs utilizing a target database system are estimated, and source statistics generation costs utilizing a source database system are estimated. The statistics generation costs are compared and statistics generation queries by the target database system are triggered in response to the statistics generation costs utilizing the target database system having a predefined relationship with the source statistics generation costs utilizing the source database system. The statistics generation queries are performed by the target database system in response to the triggering by the source database system. The generated statistics are sent from the target database system to the source database system, the table statistics in a metadata catalog are updated based on the generated statistics, and the updated table statistics are used to optimize a query plan.

Growing community of inventors

Stuttgart, Germany

Dennis Butterstein