Pivotal Software, Inc.

Emc Corporation

Salesforce, Inc.

Gopivotal, Inc.

Milena Bergant

Matthew Clark McCline

Philippe Armangau

John Michael Hayden

Daniel Zhu

Himabindu Tummala

Jeffrey Ira Cohen

Caleb E Welton

Alon Goldshuv

King Lun Choi

Raymond Angelone

Santosh Pasula Reddy

Jean-Pierre Bono

Uday K Gupta

Florian Michael Waas

Uresh K Vahalia

Sitaram D Pawar

Ajay Potnis

Jiannan Zheng

Karthikeyan Srinivasan

Konstantinos Krikellas

Suchitra Raman

Hemanth Siddulugari

Jean Pierre Bono

Hongmei Wang

Kenneth A Sell

Hefeng Yuan

Ashwin B Agrawal

Ajay S Potnis

Kenneth E Sell

Nithya Muthukrishnan

Techniques are disclosed pertaining to migrating a database between different storage clusters. A computer system identifies a current state of the database that is managed by a database application that enables a client application to access data of the database stored at a first storage cluster. The computer system copies first particular data of the database that is associated with the current state from the first storage cluster to a second storage cluster. After the copying, the computer system transitions the database application into a read-only mode in which it processes read but not write traffic. The computer system copies, from the first storage cluster to the second storage cluster, second particular data of the database that was generated by the database application during the copying of the first particular data. The computer system may also instantiate another instance of the database application in association with the second storage cluster.

Database migration

In a mirrored database system, a careful write of intentions to perform file system actions is recorded in a persistent file system objects table that is flushed to disk prior to the actions being taken. The table durably and accurately records identities of file system objects that were in use by the database to facilitate creation and deletion of physical file directories and files on a database during crash recovery and during mirror resynchronize. In the event of a failure, crash recovery may quickly and easily identify file system objects which need to be cleaned up by reference to the persistent file system objects table. Similarly, resynchronization of the mirror database can be performed quickly by referring to the persistent file system table data to detect changes since the last database checkpoint.

Persistent file system objects for management of databases

In a database cluster comprising a plurality of mirrored database segments, a mirror database pair comprising a primary database and a mirror database are maintained substantially consistent by writing changes resulting from transactions to page data in parallel to the primary database and to the mirror database. Changes are sent to the mirror database over a network using network communications. Transaction logs are also written to the primary and mirrored databases in parallel, the transaction logs being sent to the mirror database using network communications. Bulk load changes resulting from transactions are written in parallel over the network to the primary database and the mirror database without writing the changes to a transaction log. In the event of a failure, resynchronization of the mirror database can be performed quickly while avoiding a lengthy ongoing redo process on the mirror database.

Parallel database mirroring

In a mirrored database system, mirror resynchronization of bulk load and append-only tables during ongoing transactions is optimized for quick repair and high availability by catching up any changes to primary database tables that have not been made to mirror database tables during the transaction itself, and making changes due to ongoing transactions when the databases are synchronized to the resynchronization process.

Mirror resynchronization of bulk load and append-only tables during online transactions for better repair time to high availability in databases

In a mirrored database system, a careful write of intentions to perform the system actions is recorded in a persistent file system objects table that is flushed to disk prior to the actions being taken. The table durably and accurately records identities of file system objects that were in use by the database to facilitate creation and deletion of physical file directories and files on a database during crash recovery and during mirror resynchronize. In the event of a failure, crash recovery may quickly and easily identify file system objects which need to be cleaned up by reference to the persistent file system objects table. Similarly, resynchronization of the mirror database can be performed quickly by referring to the persistent file system table data to detect changes since the last database checkpoint.

A process for upgrading a mirrored shared-nothing database system comprises a sequence of short well-defined idempotent steps, and at least one non-idempotent step involving transforming a master catalog. The upgrade process is managed and controlled by a state machine that has a persistent memory running on the master node. In the event of a failure or crash during an idempotent step, the process stops the database in the current state and repeats the step. If a failure or crash occurs during a non-idempotent step, the upgrade process is rolled back to the beginning and repeated.

Mirrored database upgrade using state machine

In a mirrored database system, mirror resynchronization of fixed length pages is optimized for quick repair and high availability by identifying those lost page changes that must be made for resynchronization and ignoring other duplicate changes. Essential pages are identified using log sequence numbers (LSNs) to locate the most current version of a changed page, and other lower valued LSNs of less current versions of that page are eliminated as having changes that are duplicated in the current version of the page. Page changes since the last database checkpoint before losing mirror synchronization and the current LSN of the transaction log are identified as possibly necessary changes to be made.

Mirror resynchnronization of fixed page length tables for better repair time to high availability in databases

A method, article of manufacture, and apparatus for recovering databases. In some embodiments, this includes switching an active segment to a resynchronization mode, receiving a change tracking log, determining a data block based on the change tracking log, and resynchronizing the determined data block with a non-active segment. In some embodiments, writes performed on the active segment are synchronized with the non-active segment.

Resynchronization of nonactive and active segments

In a mirrored database system, good quality of service and high availability is afforded by minimizing the time required for suspension of operations to transition between mirror states, and by keeping track of changes made to a primary database while the mirror database was out of communications so that a later mirror resynchronization process can quickly catch up the mirror database by making only the lost changes.

Online mirror state transitioning in databases

A method, article of manufacture, and apparatus for recovering databases. In some embodiments, this includes switching an active segment to change tracking mode, intercepting write related changes on the active segment while in the change tracking mode, populating a change tracking log based on the intercepted write related changes, and storing the populated change tracking log in a storage device. In some embodiments, the storage device is memory. In some embodiments, the change tracking log if flushed from memory to disk.

Change tracking

Mirror resynchronization of fixed page length tables for better repair time to high availability in databases

Online mirror state transitions for good quality of service and high availability in databases

Growing community of inventors

San Mateo, CA, United States of America

Milena Bergant