Memsql, Inc.

Singlestore, Inc.

Adam Prout

Alex Skidanov

Marko Tintor

Ankur Goyal

Anders J Papitto

A database is partitioned into a plurality of sorted runs, wherein each sorted run includes a set of sorted segments of data records. One of the sorted runs preferably includes more than half of the records of the database, and the other sorted runs are progressively smaller. A query is processed by searching each of the sorted runs. Writes are effected by appending a new sorted run to the database. Sorted merges are used to combine the smaller sorted runs. Deletions are effected by marking the deleted record in the sorted run. Modifications are effected by deleting the original record and writing the modified record to the database. The larger sorted runs are only re-sorted or merged when the sorted run includes a substantial number of deletions. Two merge queues are maintained to enable rapid merges of the smaller sorted runs while a merger of larger sorted runs are occurring.

Fast incremental column store data loading

A set of six atomic primitives are provided for a database management system that may be used in combination to provide all of the common features and functions of a clustered database, including data failover, auto-healing, and elastic scaling. These six atomic primitives include CREATE, DROP, DETACH, ATTACH, COPY, and PROMOTE. Of particular note, it is shown that by maintaining appropriate metadata, including the status of each instance of each partition in the cluster, the versatility and reliability of this set of primitives is sufficient to implement each of the aforementioned data failover, auto-healing, and elastic scaling functions with high efficiency using a minimal number of these primitives. Each primitive is atomic (such that the cluster clearly in one state or another) and online (a workload of reads and writes is uninterrupted while the primitive runs), and each primitive is scoped to a single partition of data, thereby enabling parallel processing.

Atomic clustering operations for managing a partitioned cluster online

A system and method for executing query plans preprocesses a user's queries to identify and extract select parameters of the query, creates a skeletal query with reference to the extracted parameters, creates and compiles an executable version of the skeletal query, then executes the compiled version of the skeletal query using the particular parameters in the current query. The compiled version of the parameterized skeletal query is stored, and when another query is submitted that matches the skeletal query (but with potentially different parameters), the previously compiled skeleton query is executed with the parameters of this latter query.

Reusing existing query plans in a database system

A database durability implementation records only committed transactions in a log file. A pair of log files and a pair of snapshot files are maintained. When a snapshot of the database is completed, the 'current' log becomes the 'prior' log and the other log becomes the &#x2018;current&#x2019; log. After the next snapshot is completed, the prior log and prior snapshot may be deleted. Transactions that are not committed are not recorded in the current log, thereby avoiding the need to undo aborted transactions. If a given change is reflected in a completed snapshot, it does not appear in either of the logs; if the change is not yet reflected in a completed snapshot, it is guaranteed to be stored in one of the logs. During recovery, the system assesses both snapshots. The most recent of the completed snapshots is used, and the corresponding log(s) is (are) applied.

Growing community of inventors

San Francisco, CA, United States of America

Adam Prout