Pivotal Software, Inc.

Emc Corporation

Ivan Donald Novick

Sharad Kala

Lawrence Hamel

Goutam Tadi

Oz Basarir

Richa Sharma

Timothy Heath

Swetha Devarayasamudram Nagendran

Abhijit B Subramanya

Tim D Heath

James Bryan McAtamney

Methods, systems, and computer programs encoded on computer storage media, for implementing MPP relational databases using containers. One example system initiates primary containers to implement database segment instances. Each segment of the database is stored on a respective storage volume. Each storage volume is mounted on a respective primary container. The system detects a failure of a first primary container that is a segment instance of a first database segment. In response to the detection, the system performs a recovery process. The system unmounts, from the failed first primary container, a first storage volume storing the first database segment. The system selects a standby container from a pool of standby containers and mounts the first storage volume on the selected standby container. The system is reconfigured so that it processes queries for the first database segment using the selected standby container instead of the failed first container.

Database segment load balancer

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for allocating a number of first containers to implement one primary segment instance each and a number of second containers to implement one mirror segment instance each. In one example system, the second containers are configured to have less computing resources than the first containers. The containers are distributed among a number of physical computers. The system receives an indication of a failure of a physical computer hosting a particular first container implementing a first primary segment instance. In response to receiving the indication, the system promotes a second mirror segment instance that is a mirror of the first primary segment instance to be a new primary segment instance. The system allocates additional resources to the particular second container implementing the promoted mirror segment instance.

Light-weight mirror container

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for providing filtered backups of a distributed database. One of the methods includes receiving a user request to generate an incremental backup to be added to a user specified backup sequence for a distributed database, wherein the user specified backup sequence specifies a subset of one or more database tables to be included in backups in the user specified backup sequence. Dirty partitions of the one or more tables covered by the user specified backup sequence are identified, wherein a dirty partition is a table partition that was created or modified after generation of a most recent backup in the user specified backup sequence. An incremental backup to be added to the user specified backup sequence is generated, the incremental backup comprising contents of the dirty partitions of the tables covered by the user specified backup sequence.

Filtered incremental backups

Techniques to perform an incremental backup of a large-scale distributed database, such as a massively parallel processing (MPP) database, are disclosed. In various embodiments, a set of table partitions each of which satisfies one or more criteria to be included in a current incremental backup is determined at backup time, at least in part by comparing a current table partition meta-information with corresponding meta-information from a prior backup. The table partitions in the set are included in the incremental backup, while other table partitions not in the set are not backed up. In some embodiments, certain tables are designated to be of a type such that existing records are not modified. Tables are partitioned in a manner that isolates changes, e.g., added rows, in a relative few table partitions, leaving others unchanged and therefore able to be omitted from subsequent incremental backups.

Partition level backup and restore of a massively parallel processing database

A primary master node and a standby master node monitor the health of a shared nothing database appliance to afford high availability while minimizing false negatives and duplicate alerts by executing continuously in parallel complimentary processes that determine whether the database is running, and which master node is the active database master node. The active database master node monitors the health of the components of the database appliance by polling each component to detect failures and warnings, and the other master node monitors the status of the active master node. Upon detecting a failure of the active master node, the other node takes over health monitoring. If the database is not running, the designated primary master node performs health monitoring.

Minimizing false negative and duplicate health monitoring alerts in a dual master shared nothing database appliance

The constituency and topology of a dynamic network cluster comprising a plurality of hosts and a plurality of Level 2 network switches is automatically detected and mapped at selected intervals of time to provide current real time topology mappings. Network connections are determined by identifying the MAC addresses of host interfaces of hosts that a part of the cluster and the MAC addresses of the particular host interfaces connected to the switch ports of each switch. Resiliency to account for changes to the cluster between intervals is provided by caching cluster mapping and constituency information, and using the cached information from a previous update round to supplement information detected during a current round.

Resilient cache-based topology detection of a dynamic cluster

A distributed reusable framework comprising a non-logging resumable workflow of actions that runs independently in parallel to upgrade each machine of a large distributed cluster of machines. A CHECK process and a RUN process are associated with each action in the workflow. The CHECK process determines dynamically the state of each action, and the RUN process brings an action to a completed state. Following a reboot on a machine, the workflow on the machine sequences back to its start and rechecks each prior action of the workflow to verify that each action is in a completed state before. Upon a failure of the upgrade on a single machine in the cluster, the failed machine can be fixed and the workflow can be resumed without redoing the work already done. Since the actual live state is verified on each machine of the cluster, log files are unnecessary.

Growing community of inventors

Sunnyvale, CA, United States of America

Ivan Donald Novick