Microsoft Technology Licensing, LLC

Linkedin Corporation

Srinath Shankar

Scott M Meyer

Alan D Halverson

Rimma V Nehme

Robert W Stephenson

Jeffrey F Naughton

Andrew Joseph Carter

Eric R Robinson

Nikhil Teletia

Huaxin Liu

Hideaki Kimura

Willis Lang

Walaa Eldin M Moustafa

Karthik Saligrama Ramachandra

Karan Rajen Parikh

Maverick Shu-Bun Lee

Shyam Shankar

Yejuan Long

Andrew J Carter

Distributed query processing is often performed by a set of nodes that apply MapReduce to a data set and materialize partial results to storage, which are then aggregated to produce the query result. However, this architecture requires a preconfigured set of database nodes; can only fulfill queries that utilize MapReduce processing; and may be slowed down by materializing partial results to storage. Instead, distributed query processing can be achieved by choosing a node for various portions of the query, and generating customized code for the node that only performs the query portion that is allocated to the node. The node executes the code to perform the query portion, and rather than materializing partial results to storage, streams intermediate query results to a next selected node in the distributed query. Nodes selection may be involve matching the details of the query portion with the characteristics and capabilities of the available nodes.

Per-node custom code engine for distributed query processing

The disclosed embodiments provide a system for processing queries of a graph database. During operation, the system executes one or more processes for providing the graph database storing a graph, wherein the graph includes a set of nodes, a set of edges between pairs of nodes in the set of nodes, and a set of predicates. Next, the system receives a query containing a missing value parameter, wherein the missing value parameter designates an attribute in the graph database with a nonexistent value. The system then produces, from the query, a transformed query comprising one or more parameters that do not include the missing value parameter. Finally, the system uses the transformed query and the graph database to generate a result of the query and provides the result in a response to the query.

Transforming and evaluating missing values in graph databases

The present invention extends to methods, systems, and computer program products for formulating global statistics for parallel databases. In general, embodiments of the invention merge (combine) information in multiple compute node level histograms to create a global histogram for a table that is distributed across a number of compute nodes. Merging can include aligning histogram step boundaries across the compute node histograms. Merging can include aggregating histogram step-level information, such as, for example, equality rows and average range rows (or alternately equality rows, range rows, and distinct range rows), across the compute node histograms into a single global step. Merging can account for distinct values that do not appear at one or more compute nodes as well as distinct values that are counted at multiple compute nodes. A resulting global histogram can be coalesced to reduce the step count.

Optimizing queries using global statistics for distributed databases

The present invention extends to methods, systems, and computer program products for optimizing parallel queries using interesting distributions. For each logical operator in an SQL server MEMO, in a top down manner from a root operator to the leaf operators, interesting distributions for the operators can be identified based on the properties of the operators. Identified interesting distributions can be propagated down to lower operators by annotating the lower operators with the interesting distributions. Thus, a SQL server MEMO can be annotated with interesting distributions propagated top down from root to leaf logical operators to generate an annotated SQL server MEMO. Parallel query plans can then be generated from the annotated SQL server MEMO in a bottom up manner from leaf operators to a root operator. Annotated interesting properties can be used to prune operators, thereby facilitating a more tractable search space for a parallel query plan.

Optimizing parallel queries using interesting distributions

In order to facilitate the requesting of desired information from a graph database that stores a graph, a system may translate an initial query into a query that can be executed against the graph database. In particular, using primitives, the system may translate a query associated with a type of database (such as a relational database) into the query. The graph may include nodes, edges between the nodes, and predicates to represent and store data with index-free adjacency. Moreover, the primitives may include: a rule based on edges in the graph that expresses a relational schema in the type of database, and information associated with a compound key that specifies a relationship between nodes, edges and predicates in the graph corresponding to a table in the type of database. Then, the system may execute the query against the graph database, and may receive a result that includes a subset of the graph.

Translating queries into graph queries using primitives

In order to request desired information from a graph database, a system executes a query against the graph database. The graph database may store a graph that includes nodes, edges between the nodes, and predicates to represent and store data with index-free adjacency. Moreover, the query may identify a first edge associated with a predicate that specifies one or more of the nodes in the graph. In response to the query, the system receives a result that includes a subset of the graph. In particular, the subset of the graph may include the desired information expressed within an associated structure of the graph.

Graph-based queries

In order to request desired information from a graph database, a system executes a query against the graph database. The graph database may store a graph that includes nodes, edges between the nodes, and predicates to represent and store data with index-free adjacency. Moreover, the query may identify a first edge associated with a predicate that specifies one or more of the nodes in the graph. In response to the query, the system receives a result that includes a subset of the graph. Then, the system may use the result of the query in a concatenated set of queries. In particular, the system may execute a second query against the subset of the graph. Furthermore, in response to the second query, the system may receive a second result that includes a second subset of the graph.

Concatenated queries based on graph-query results

In order to facilitate efficient storage and extraction of data, a system may represent a compound relationship in a graph stored in a graph database. In particular, the system (or a user) may define the compound relationship based on two or more predicates associated with two or more edges between two or more nodes in the graph. Then, the system may generate, in the graph, a hub node that corresponds to the compound relationship and that has an identifier that is hidden from users of the graph so that the identifier of the hub node is not visible external to the graph database. Moreover, the identifier of the hub node may be specified in a query using the two or more predicates, thereby allowing data associated with the compound relationship to be readily identified, modified and/or extracted.

Representing compound relationships in a graph database

In order to request desired information from a graph database, a system executes a query against the graph database. The graph database may store a graph that includes nodes, edges between the nodes, and predicates to represent and store data with index-free adjacency. Moreover, the query may identify a first edge associated with a predicate that specifies one or more of the nodes in the graph. In response to the query, the system receives a result that includes a subset of the graph. Then, the system may verify the subset of the graph. In particular, the system may verify that the subset of the graph includes one or more particular edges, such as an edge with the predicate in the query. Furthermore, the system may verify that the subset of the graph includes data and/or associated structure in the portion of the graph.

Verifying graph-based queries

Presented is a system and method for evaluating relational database queries in a distributed system. An optimized query plan is received by a control node. The query plan is decoded to a SQL statement that is semantically equivalent to the query plan, even though the query plan contains elements that have no direct analog in SQL. The decoded SQL is transmitted to a compute node for execution.

SQL generation for assert, update and delete relational trees

Distributing rows of data in a distributed table distributed across a plurality of nodes. A method includes identifying skewed rows of a first table to be distributed in a distributed database system. The skewed rows include a common data value in a column such that the skewed rows are skewed, according to a predetermined skew factor, with respect to other rows in the first table not having the common data value. Non-skewed rows of the first table that are not skewed according to the skew factor are identified. The skewed rows of the first table are distributed across nodes in a non-deterministic fashion. The non-skewed rows of the first table are distributed across nodes in a deterministic fashion. The rows of the first table distributed across the nodes, whether distributed in a deterministic fashion or non-deterministic fashion, are stored in a single table at each of the nodes.

Skew-aware storage and query execution on distributed database systems

A single system image is provided for a parallel data warehouse system by exposing a shell database within a database management system comprising metadata and statistics regarding externally stored data. Further, functionality of the database management system can be exploited to perform pre-execution tasks. In one instance, one or more execution plans can be generated by the database management system for an input command and subsequently employed to generate a distributed execution plan.

Growing community of inventors

Madison, WI, United States of America

Srinath Shankar