International Business Machines Corporation

Nirmal Kumar

Pravin Kailashnath Kedia

James D Spyker

Gaurav Mehrotra

Elaine Pang

Dev K Sarkar

Hrishikesh Sujaya Kumar

Girish V Mattur

Phillip Koza

Pratik P Paingankar

Venkat R Arra

Rashmi Bharadwaj, V

Jagadisha Bhat

In optimizing net change data replication across a plurality of transactions in a replication environment, a source replication engine at a source database system: obtains a window in a source database log for the net change data replication; for a given row identifier found in the source database log, identifies an initial operation and a latest operation across the plurality of transactions within the window; compares the initial operation and the latest operation for the given row identifier to determine a net change to a row associated with the given row identifier; and stores the net change as a final operation on the given row identifier in an optimization repository for replication to a target database. The operations stored in the optimization repository are then sent to a target database system for replication to the target database.

Net change mirroring optimization across transactions in replication environment

A computer-implemented method includes writing a plurality of change records to a first set of redo records in a first transaction log file. The computer-implemented method further includes determining, while writing a change record to a redo record, that the change record includes changes made to data associated with an in-scope table. The computer-implemented method further includes updating the first transaction log file with information indicating that the redo record includes changes made to data associated with the in-scope table. The computer-implemented method further includes accessing the first transaction log file from storage in response to receiving a request for replicating changes made to data associated with in-scope tables. The computer-implemented method further includes scraping a first redo record in the first transaction log file in response to determining that the first redo record includes at least one change record associated with the in-scope table.

Asynchronous replication of in-scope table data

A dynamically self-indexing database-management system selects database indexes associated with table columns that are most frequently accessed by user queries, deleting all other indexes. The system periodically reviews database-performance figures and data-usage patterns for each table of its database and revises its selection of indexes in order to ensure that only the most frequently accessed columns continue to be indexed and that the omission of other indexes does not degrade performance. The total number of selected indexes, the overall percent of selected indexes, or the selection itself is optimized over time through continued monitoring of database transaction logs. Optimization may comprise cognitive analytics or other methods of artificial intelligence by which the system learns over time how to best determine whether its current selection of indexes is likely to provide the best overall performance.

Dynamically self-indexing database-management system

A system and method for performing a dynamic rollover of source and target latency in a replication environment. One or more tables of a source data store being replicated to a target data store are parked, wherein any subsequent changes for the parked tables are not replicated to the target data store. Open transactions are identified where the parked tables affect the source and target latency, wherein the open transactions are in-progress and not completed. The source and target latency are then calculated for the identified open transactions.

Dynamic roll-over of source and target latency in a replication environment

In an approach for database log management and log access performance improvement, a processor receives an access request for an optimal control file to minimize a database access performance impact. A processor determines the optimal control file based on usage of a plurality of control files for a database. A processor outputs the optimal control file to a user to access the database. A processor provides an incremental sub-control file of the optimal control file to the user. The incremental sub-control file is an incremental view of the optimal control file to query the incremental log data in the database.

Database access performance improvement

Transaction change data replication includes identifying changes being made to a source database as part of an ongoing transaction at a source. The identifying is performed as the changes are made to the source database and as the transaction remains ongoing prior to commit or rollback thereof at the source. The source and a target are in a replication relationship in which data of the source database at the source is replicated to destinations in a target database at the target. The indications of the changes being made to the source are forwarded, to the target, as the transaction remains ongoing prior to commit or rollback thereof, and based on ending the transaction at the source, an indication of the transaction end is sent to the target.

Transaction change data forwarding

A method, computer program product, and computer system for handling open transactions in a data replication environment is provided. The method includes determining a database user that initiated an earliest open transaction in a transaction log. The method further includes, upon determining that the database user does not have access privileges for one or more tables of a target database corresponding to one or more tables of a source database associated with the earliest open transaction, skipping replication of the earliest open transaction to the target database, such that the data replication system no longer waits for the earliest open transaction to complete. The skipping includes discarding, from the transaction log, one or more entries related to the earliest open transaction, and identifying, in the transaction log, a next earliest open transaction.

ACL based open transactions in replication environment

Log reader and parser sharing determination in a change data capture environment includes obtaining log record identifiers for a group of data consumers of transaction log file data in transaction log files, identifying a respective restart log position and current log position for each data consumer of the group of data consumers, determining transaction log file distances between the restart log positions and the current log positions of each of one or more pairs of data consumers, calculating resource costs for data consumers of each pair to share a log reader and log parser, determining whether to share a log reader and log parser between any two or more data consumers of the group, and performing processing based on the determining whether to share.

Log reader and parser sharing determination in a change data capture environment

Embodiments generally relate data replication in databases. In some embodiments, a method includes accessing transaction information from a staging store at a first database. The method further includes determining one or more records to be prefetched from a hard disk of a second database based on the transaction information. The method further includes prefetching the one or more records to be prefetched from the hard disk. The method further includes storing the one or more prefetched records in a memory of the second database. The method further includes updating the one or more records stored in the second database based on the transaction information.

Data replication in a database environment

An approach for inter-table parallel refresh in a database management system (DBMS). An inter-table refresher receives DBMS source and target databases comprising source parent and related source child datasets. The inter-table refresher determines a collection of the source parent and the related source child datasets and creates a collection of parent and child refresh segments based on the collection of the source parent and child datasets. The inter-table refresher creates a collection of parent and child refresh batches based on the collection of parent and child refresh segments respectively and outputs an inter-table refresh batch schedule based on the collection of the parent refresh batches and the collection of the child refresh batches where the collection of the parent refresh batches and the collection of the child refresh batches operate concurrently and a current child refresh batch follows the completion of a respective current parent refresh batch.

Inter-table parallel refresh maximizer

A processor-implemented method for selective data replication recovery in a heterogeneous environment is provided, including a source agent, a target agent, and data replication recoverer (DRR) agent. The method includes receiving, by a DRR agent, one or more committed transaction records from a source agent, which is configured to receive the one or more committed transaction records from a source database. The DRR agent creates data and metadata records from the received one or more committed transaction records, and saves the data and the metadata records in a data replication repository. In response to receiving a request to recover a target database, the DRR agent selectively recovers target database using either one or more individual transactions or a bookmark.

Smart data replication recoverer

A method for handling of open transactions in a data replication environment. The method includes querying one or more tables of a source database to determine a first open transaction associated with the one or more tables of the source database. The method further includes determining a database user that initiated the first open transaction. The method further includes querying one or more tables of the target database corresponding to the one or more tables of the source database associated with the first open transaction, to determine whether the database user has access privileges for the one or more tables of the target database and dynamically skipping replication of the first open transaction to the target database, such that the data replication system no longer waits for the first open transaction to complete.

Growing community of inventors

Bengaluru, India

Nirmal Kumar