Microsoft Technology Licensing, LLC

Microsoft Corporation

Per-Ake Larson

Cristian Diaconu

Michael James Zwilling

Robert P Fitzgerald

Jingren Zhou

Craig Steven Freedman

Justin Jon Levandoski

Jonathan David Goldstein

Cesar Alejandro Galindo-Legaria

Srikumar Rangarajan

Jose A Blakeley

Hanumantha Rao Kodavalla

Artem A Oks

Ahmed Eldawy

Mike Zwilling

Yi-Min Wang

Atul Adya

Goetz Graefe

Oliver H Foehr

Lili Qiu

Murali R Krishnan

Donald Kossmann

Ronnie Chaiken

Aleksandras Surna

Chad E Verbowski

Guido Moerkotte

Peter Alan Carlin

Sergey Melnik

Demetrios Achlioptas

George V Reilly

Gautam Das

Spyridon Blanas

Umar Farooq Minhas

David Edward DeHann

Tianzheng Wang

Joy James Prabhu Arulraj

Dimitrios Tsirogiannis

Donghui Zhang

Frank W Tompa

Robert Patrick Fitzgerald

Cristian C Diaconu

Robert Patrick Fitgerald

Various technologies described herein pertain to accessing records of an in-memory database. A memory can include a primary storage and a data store can include a secondary storage. The primary storage includes a first subset of the records of the database, and the secondary storage includes a second subset of the records of the database. Moreover, the memory includes an update memo that includes timestamp notices. The timestamp notices specify statuses of records of the database having updates that span the primary storage and the secondary storage. The memory further includes a database management system that is executable by a processor. The database management system processes a transaction. Further, the database management system includes a storage interface component that accesses a record of the database for the transaction based on at least one of the timestamp notices of the update memo.

Transactional access to records on secondary storage in an in-memory database

An in-memory database system stores an entirety of a database within main memory of the computer system. The database includes a table comprising a plurality of rows including a particular row. A first transaction updating the particular row is executed to create a first version of the particular row. A second transaction updating the particular row is executed to create a second version of the particular row. The first version is visible only to the first transaction, and the second version is visible only to the second transaction. The in-memory database system may associate an index with the database table, the index referencing the plurality of rows, including referencing the first version of the particular row and the second version of the particular row. The in-memory database system may initiate garbage collection of the first version of the particular row and/or the second version of the particular row.

In-memory database system

The subject disclosure relates to a distributed transaction management technique that ensures synchronization between participating nodes in a global or distributed database transaction. The technique leverages a commit protocol that uses local clocks at the respective participating nodes. Participants in a global transaction are configured to utilize the same commit timestamp and logical read time and can advance their respective local clocks to establish this synchronization. In one embodiment, distributed commit utilizes a modified version of two-phase commit that includes an extra phase to collect commit timestamp votes from participants. Additionally, a heartbeat mechanism can be used to establish loose synchronization between nodes. In another embodiment, a node can respond to a remote transaction request by returning a list of nodes involved in generating the result of the transaction and the types of access used by such nodes in addition to the transaction result itself.

Distributed transaction management

A versioned records management computing system that uses a restart era in order to promote rapid recovery. A persistent store includes a multi-versioned record collection. The records are also associated with a restart era that corresponds to the era of operation of the computing system after a restart. Upon a recovery, the current restart era changes. An object collection media has an object collection that conforms to an object model such that the object model is used to operate upon the records. The object collection media is operable such that the object collection is durable so as to survive restarts of the system to thereby allow for accelerated recovery.

Versioned records management using restart era

A versioned records management system that uses a restart era in order to promote rapid recovery. A persistent store includes a plurality of items that are each marked with a restart era. Any item that was created in the current restart era is marked with the current restart era. The persistent store also includes a first subset of the items comprising a plurality of records. At least one record includes multiple corresponding versions. The persistent store also includes a second subset of the items comprising a collection of objects that conform to an object model that is used to operate upon the plurality of records via the object collection. The versioned records management system recovers from a crash by promoting an item in the persistent store to the current restart era, in parallel with normal operation of the versioned records management system.

A log manager causing events to be written to the primary and the secondary computing systems for high availability, and returning threads quickly. Rather than cause every event write to be immediately written to the secondary computing system, the log management component is triggered to write to the secondary computing system upon detecting event write requests of event writes of a particular event type. In response to detecting such an event write, the log management component gathers a group of event writes corresponding to that particular event write, and dispatches the event writes as a group. The particular thread that initiated the particular event write of the particular event type is returned after dispatching the event write group, but before or without confirmation of the particular event write group has been written to the secondary computing system.

Early thread return with secondary event writes

The recovery of one or more transactions in a computing system in which one or more persistent systems are coupled via a bus to one or more processors. As an example, the persistent system may serve as at least part of the main memory of the computing system. The transaction might implement multi-versioning in which a record is not updated in place. Rather, each record is represented as a sequence of one or more record versions, each version having a valid interval during which the record version is considered to properly represent the record.

Recovery processing for database in persistent system

A computer implemented method includes receiving multiple requests to update a data structure stored in non-volatile memory (NVM) and applying an atomic multiword update to the data structure to arbitrate access to the NVM. In a further embodiment, a computer implemented method includes allocating a descriptor for a persistent multi-word compare-and-swap operation (PMwCAS), specifying targeted addresses of words to be modified, returning an error if one of the targeted addresses contains a value not equal to a corresponding compare value, executing the operation atomically if the targeted addresses contain values that match the corresponding compare values, and aborting the operation responsive to the returned error.

Index structure using atomic multiword update operations

Methods, systems, and computer-readable media of columnar storage of a database index are disclosed. A particular columnar index includes a column store that stores rows of the columnar index in a column-wise fashion and a delta store that stores rows of the columnar index in a row-wise fashion. The column store also includes an absence flag array. The absence flag array includes entries that indicate whether certain rows have been logically deleted from the column store.

Columnar storage of a database index

The subject disclosure relates to a database recovery technique that implements various aspects of transaction logging to optimize database recovery performance. Transactions are logged logically with no reference to physical location, which enables logging to be performed via multiple independent log streams in parallel. A set of log streams can further be adjusted to conform to a local configuration of a mirror or secondary node in order to increase mirroring flexibility. Additionally, individual transactions or groups of transactions are recorded using a single log record, which contains timestamp information to enable database recovery without reference to physical checkpoint files. Further, techniques are provided herein for processing transactions without Write Ahead Logging or hardening of undo information. As further described herein, a transaction can be committed via a set of hierarchical stages, which in turn can facilitate integration of an in-memory database system with one or more external database systems.

Versatile in-memory database recovery using logical log records

A system that uses a persistent main memory to preserve events that await logging in a persistent store. Each event is written into the persistent main memory so as to be loggable in case of recovery. For instance, the event may be written into a log cache structure, along with other state which identifies that the event is in the particular log cache structure, the location of the event within the particular log cache structure, and the order of the event. To recover, the log in the persistent store is evaluated to identify the end of the stored log. The tail of the log is identified in the persistent main memory by identifying any log cache structures that are after the end of the stored log and which are validly recoverable. The log cache structure contents are then serialized one log cache at a time, earliest first.

Tail of logs in persistent main memory

An in-memory database system stores the entirety of a database that is being operated on by a database system within main memory. The database includes a table that comprises a plurality of rows including a particular row. The database table comprises a first version of the particular row having a first valid time, and a second version having a second valid time. Index(es) are associated with the database table. Each index references the plurality of rows, including referencing the first and second versions of the particular row. A transaction acting on the first version of the particular row is executed. The first version of the particular row is visible to the transaction based on the first valid time and the second version of the particular row being not visible to the transaction based on the second valid time.

Performing a transaction in the context of a computing system that has one or more persistent systems coupled to one or more processors over a bus. As an example, the persistent system may serve as at least part of the main memory of the computing system. The transaction might implement multi-versioning in which a record is not updated in place. Rather, each record is represented as a sequence of one or more record versions, each version having a valid interval during which the record version is considered to properly represent the record. The transaction processing uses torn write detection so that recovery processes may use such guards to verify that there are no torn writes. For instance, torn write guards may be used to verify the integrity of record versions as well as the log buffers that refer to the record versions.

Transaction processing using torn write detection

Growing community of inventors

Redmond, WA, United States of America

Per-Ake Larson