Microsoft Technology Licensing, LLC

Craig Steven Freedman

Michael James Zwilling

Daniel Georg Schall

Cristian Diaconu

Hanumantha Rao Kodavalla

Panagiotis Antonopoulos

Per-Ake Larson

Naveen Prakash

Erik Ismert

Raghavendra Thallam Kodandaramaih

Adrian-Leonard Radu

Alejandro Hernandez Saenz

Conor J Cunningham

Chandrashekhar Kadiam

Goetz Graefe

Cesar Alejandro Galindo-Legaria

Milind Madhukar Joshi

Gang He

Vasileios Papadimos

Prashanth Purnananda

Devin Leo Rider

Stefano Stefani

Girish Mittur Venkataramanappa

Peter Byrne

Wei Ting Chen

Raul Garcia

Jack Smith Richins

Nitish Jindal

Marcel Van Der Holst

Milos Andelković

Marko Zivanović

Nikola Vujic

Gargi Sur

Nikola Vujić

Chaitanya Sreenivas Ravella

Devin L Rider

Methods, systems, and computer program products are provided that construct and execute multi-step query plans in a database application. A database application accepts a query such as, for example, a graph query. The database application generates a physical query plan comprising a set of execution steps that perform the query. The set of execution steps comprises at least an initial step, an intermediate step and a final step. The database executes the query by passing control to the initial step that, when complete, passes execution control to some other step. The intermediate step executes and when complete, may pass execution control to any other step including itself. Steps may be configured to communicate arbitrary data between steps. The generated query plan may also comprise multiple multi-step sequences, and such sequences may be configured to execute in the alternative based on intermediate query results, or a parameter included in the query.

Multi-step query execution in SQL server

Methods for operation fragmentation with metadata serialization in query processing pushdowns are performed by systems and devices. A compute node receives a query directed to database data, and generates query text fragments. Portions of metadata of the database are read from different page servers, and are serialized by the compute node. Page identities of data pages in a page server that stores the data are determined from a page index at the compute node, and the compute node provides the text fragments, the serialized metadata, and the page identities to the page server storing the data. The page server compiles the text fragments based on the serialized metadata to generate an executable query plan for the query. The page server initializes and performs execution of the executable query plan against the data as a pushdown query operation, and a result of pushdown query operation is provided to the compute node.

Operation fragmentation with metadata serialization in query processing pushdowns

Distributed database systems including compute nodes and page servers are described herein that enable compute nodes to pushdown certain query processing compute tasks to the page servers to take advantage of otherwise idle compute resources at the page servers, and to reduce the quantity of data that moves between compute nodes and page servers. A distributed database system includes a page server and a compute node, wherein the page server is configured to maintain multiple versions of stored data objects. The compute node is configured to receive a query and generate a transaction context (TC) and modified table schemas (MTS) scoped to the query, and pushdown the query, TC and MTS to the page server that is configured to determine which data objects at the page server satisfy the query, and for each such object, which version of the object should be returned based on the TC.

Schema and data modification concurrency in query processing pushdown

Example aspects include techniques for anomaly detection via sparse judgmental samples. These techniques may include generating a qualified timestamp corresponding to execution of a first operation of a first transaction over a database and determining, based on the qualified timestamp, that a row of a table is not locked by a second operation of a second transaction over the database. In addition, the techniques may include determining that the row meets a condition of the first operation. Further, the techniques may include updating, based on the qualified timestamp, the row in response to the row meeting the condition, the updating including acquiring a lock on the row of the table.

Method and system for lock after qualification for update queries

Methods for page split detection and affinity in query processing pushdowns are performed by systems and devices. Page servers perform pushdown operations based on specific, and specifically formatted or generated, information, instructions, and data provided thereto from a compute node. Page servers also determine that page splits have occurred during reading of data pages maintained by page servers during pushdown operations, and also during fulfillment of compute node data requests. To detect a data page has split, page servers utilize information from a compute node of an expected next data page which is compared to a next data page in the page server page index. A mismatch in the comparison by page servers indicates data page was split. Compute nodes and page servers store and maintain off-row data generated during data operations via page affinity considerations where the off-row data is stored at the same page server as the data.

Page split detection and affinity in query processing pushdowns

A computing device is provided, including non-volatile memory storing a database including a table having a plurality of rows. The computing device may further include a processor configured to receive a request to perform a first transaction on a row. The processor may assign a first transaction identifier (TID) of the first transaction to the row. The processor may impose a first exclusive TID lock on the first TID of the row and may impose a first exclusive data lock associated with the first transaction on the row. The processor may perform the first transaction on the row. Performing the first transaction may include modifying the table as stored in the memory. In response to completing the first transaction, the processor may release the first exclusive data lock on the row and release the first exclusive TID lock on the first TID of the row.

Transaction identifier locking with data row locks

Examples described herein generally relate to executing a received graph database query. The received graph database query can be converted into a recursive common table expression (CTE). Multiple alternative processes for executing the recursive CTE can be generated based on the recursive CTE. A cost associated with each of the multiple alternative processes can be determined. One of the multiple alternative processes can be converted into a multi-step sequence based on the associated cost. The multi-step sequence can be executed on a database to retrieve a set of results in response to the received graph database query.

Techniques for optimizing graph database queries

An in-memory database system stores an entirety of a database within main memory of the computer system. The database includes a table comprising a plurality of rows including a particular row. A first transaction updating the particular row is executed to create a first version of the particular row. A second transaction updating the particular row is executed to create a second version of the particular row. The first version is visible only to the first transaction, and the second version is visible only to the second transaction. The in-memory database system may associate an index with the database table, the index referencing the plurality of rows, including referencing the first version of the particular row and the second version of the particular row. The in-memory database system may initiate garbage collection of the first version of the particular row and/or the second version of the particular row.

In-memory database system

A database system includes a query optimizer which applies transformations to a logical representation of an input query in a declarative query language to generate expressions for a query execution plan. The query optimizer stores information, for each rule application, indicating a transformation applied and bindings for the transformation, the bindings indicating expressions input to the transformation. When a new query execution plan is to be generated for the input query, the query optimizer uses this stored information, for expressions in an original query execution plan, to guide the query optimizer to produce a current query execution plan while avoiding transformations not used to generate expressions in the original query execution plan.

Recording lineage in query optimization

A data system in which a parent transaction that has multiple nested transactions has isolation between the nested transactions. Each of at least some of the plurality of operations are included within a nested transaction within the parent transaction. For each of at least some of the nested transactions, a begin transaction identifier is assigned to the nested transaction; and on or around completion of the nested transaction, a completion transaction identifier is assigned as a statement identifier at least one object version operated upon by the nested transaction. Visibility of a particular version of an object to an active nested transaction may be verified by comparing the begin transaction identifier of the active nested transaction to the statement identifier of the particular version of the object. The comparison is done such that visibility achieves the proper isolation between nested transactions.

Multi-version data system nested transactions isolation

Running a transaction against a database that allows the transaction to be automatically retried when the transaction fails such that certain transaction failures that cause the transaction to be automatically retried are not visible to an entity that receives data as a result of the execution of the transaction. A first set of statements of a transaction is identified by identifying statements that will cause a change in the database state. A second set of statements is identified, that result in data being generated to be sent to the entity. The method further includes executing the first set of statements to change the database. The method further includes running the second set of statements separately from the first set of statements to generate data to be sent to the entity.

Automatically retrying transactions with split procedure execution

A system and method of enabling row level security through security policies is disclosed herein. In this system and method, a computing device may be communicatively coupled to a storage device. The computing device may further be activated and maintain data that comprises a plurality of rows. When executed by the computing device, the system and method may process a data definition language statement comprising a security policy definition. Further, the system and method may receive a query language statement comprising a request to access a first column of a row from the plurality of rows. The system and method may process the request and determine if access may be granted to a user based on the security policy definition in the system.

Row level security

Growing community of inventors

Sammamish, WA, United States of America

Craig Steven Freedman