Microsoft Technology Licensing, LLC

Raghavendra Thallam Kodandaramaih

Hanumantha Rao Kodavalla

Panagiotis Antonopoulos

Adrian-Leonard Radu

Craig Steven Freedman

Naveen Prakash

Alejandro Hernandez Saenz

Daniel Georg Schall

Girish Mittur Venkataramanappa

Peter Byrne

Cristian Diaconu

Wei Ting Chen

Arkadi Brjazovski

Rogerio Ramos

Daniel Vasquez Lopez

Christian Damianidis

Varun Kunjbihari Tibrewal

Prashanth Purnananda

Chaitanya Sreenivas Ravella

Wei Chen

Methods for operation fragmentation with metadata serialization in query processing pushdowns are performed by systems and devices. A compute node receives a query directed to database data, and generates query text fragments. Portions of metadata of the database are read from different page servers, and are serialized by the compute node. Page identities of data pages in a page server that stores the data are determined from a page index at the compute node, and the compute node provides the text fragments, the serialized metadata, and the page identities to the page server storing the data. The page server compiles the text fragments based on the serialized metadata to generate an executable query plan for the query. The page server initializes and performs execution of the executable query plan against the data as a pushdown query operation, and a result of pushdown query operation is provided to the compute node.

Operation fragmentation with metadata serialization in query processing pushdowns

Distributed database systems including compute nodes and page servers are described herein that enable compute nodes to pushdown certain query processing compute tasks to the page servers to take advantage of otherwise idle compute resources at the page servers, and to reduce the quantity of data that moves between compute nodes and page servers. A distributed database system includes a page server and a compute node, wherein the page server is configured to maintain multiple versions of stored data objects. The compute node is configured to receive a query and generate a transaction context (TC) and modified table schemas (MTS) scoped to the query, and pushdown the query, TC and MTS to the page server that is configured to determine which data objects at the page server satisfy the query, and for each such object, which version of the object should be returned based on the TC.

Schema and data modification concurrency in query processing pushdown

Methods for page split detection and affinity in query processing pushdowns are performed by systems and devices. Page servers perform pushdown operations based on specific, and specifically formatted or generated, information, instructions, and data provided thereto from a compute node. Page servers also determine that page splits have occurred during reading of data pages maintained by page servers during pushdown operations, and also during fulfillment of compute node data requests. To detect a data page has split, page servers utilize information from a compute node of an expected next data page which is compared to a next data page in the page server page index. A mismatch in the comparison by page servers indicates data page was split. Compute nodes and page servers store and maintain off-row data generated during data operations via page affinity considerations where the off-row data is stored at the same page server as the data.

Page split detection and affinity in query processing pushdowns

A computing device is provided, including non-volatile memory storing a database including a table having a plurality of rows. The computing device may further include a processor configured to receive a request to perform a first transaction on a row. The processor may assign a first transaction identifier (TID) of the first transaction to the row. The processor may impose a first exclusive TID lock on the first TID of the row and may impose a first exclusive data lock associated with the first transaction on the row. The processor may perform the first transaction on the row. Performing the first transaction may include modifying the table as stored in the memory. In response to completing the first transaction, the processor may release the first exclusive data lock on the row and release the first exclusive TID lock on the first TID of the row.

Transaction identifier locking with data row locks

A relational database system that implements persistent version storage may include an in-row module that is executable by one or more processors to implement an in-row scheme by storing in-row previous version information within a payload of a row within a database page. The relational database system may additionally include an off-row module that is executable by the one or more processors to implement an off-row scheme by storing off-row previous version information in an off-row page that is separate from the database page. The relational database system may additionally include a storage policy that defines when previous version information is stored in accordance with the in-row scheme and when the previous version information is stored in accordance with the off-row scheme. The relational database system may additionally include a cleanup module that cleans up older versions when they are deemed unnecessary by the system.

Persistent version storage for relational database management system

A method for supporting concurrent updates to a database page may include providing shared access to the database page. A first update may be performed with respect to a first record in the database page. A second update may be performed with respect to a second record in the database page. The first update and the second update may be performed concurrently while the shared access to the database page is being provided. The method may also include synchronizing the first update and the second update.

Supporting concurrent updates to a database page

Database recovery can be performed in substantially constant time. From a database transaction log, the state of one or more transactions is identified including those that were active at a time of a crash but not committed. Transactions can be reapplied sequentially starting from the beginning of a last successful checkpoint to the end of the transaction log. Uncommitted user transactions are not undone synchronously, but simply marked as aborted. Subsequently, versions of affected data elements can be reverted back to a prior saved version asynchronously.

Constant time database recovery

The insertion into a data stream of collection records associated with multiple collections of data items in the data stream, for purposes of aiding in subsequent processing of the data stream by collections. Each collection record comprises a collection definition that does not overlap with the collection definition in any of the sequence of collection records including the collection definition of any neighboring collection record in the sequence of collection records. Each collection record also includes a data stream address range extending from the first data item to the last data item in the collection.

Collection record for overlapping data stream collections

The logging of tasks associated with multiple transactions in the context in which the computing system is capable of having multiple of the transactions active at a given time, and thus in which the order of completion of the transactions is not guaranteed to be the same as the order of initiation of the transactions. A sequence of transaction segment records is added to the log. Each transaction segment record comprises a transaction identifier range that does not overlap with the transaction identifier range in any of the sequence of transaction segment records including the transaction identifier range of any neighboring transaction segment record in the sequence of transaction segment records. Each transaction segment record also includes a log entry identifier range extending from the first task of the first of the transactions to be initiated to the last task of the last transaction to complete.

Collection record location as log tail beginning

The use of a data stream that has therein data items and a sequence of collection records. each comprising a collection definition that is not overlapping with the collection definition in any of the sequence of collection records. The collection definition defines which data items of the data stream are included within the collection. Each collection record also includes a data stream address range at least extending from the data stream address of the first data item of the collection to the data stream address of the last data item in the collection. In this context, the data stream may be efficiently processed by quickly reviewing the data stream to find each collection record. Once a collection record is found, the collection record is dispatched for processing to a worker thread for processing of the corresponding collection.

Growing community of inventors

Redmond, WA, United States of America

Raghavendra Thallam Kodandaramaih