Microsoft Technology Licensing, LLC

Clock Synchronization

Anders Tungeland Gjerdrum

Jan-Ove Almli Karlberg

Tor Kreutzer

Richendra Khanna

Dushyanth Narayanan

Miguel Castro

Aleksandar Dragojevic

Alexander Shamis

Stanko Novakovic

Matthew Renzelmann

Georgios Chatzopoulos

Theodoros Gkountouvas

Iqra Ali

Anthony Acquah

Ronny Jensen

Kai-Marius Pedersen

Vincent Macharia Karuri

Vincent Mulwa Kyalo

Marius Indreberg

Technology is disclosed for programmatically parallelizing distributed graph queries of a graph metaphor of distributed data sources through various applications or platforms. Query candidates corresponding to distributed data sources are determined by applying a graph query to a graph metaphor of the distributed data sources. A set of query steps representing a set of distributed queries of the query candidates are determined based on corresponding properties of the query candidates from the graph metaphor. The set of query steps are executed in parallel in order to determine a response to the graph query. The data is provided in response to the graph query.

Parallelization of distributed graph queries

Solutions are disclosed that enable efficient federated graph queries across multiple isolated data stores. Examples leverage the connectedness of the expected data that spans the data stores by defining the entities and relationships and inferring the intent of the queries. These are used to optimize data searches in the individual data stores. Examples map each of two or more variables of the input query to elements of a public schema and use the mapping to determining a storage tag (identifying a data store) for each of the variables of the input query. Store-specific queries are scheduled and performed based on at least the storage tags.

Federated graph queries across heterogeneous data stores

Example solutions for performing federated graph queries use schemas and include: generating a logical graph using a query input and generating a physical plan using a query planning schema for double-nested loops, the outer loop for each permutation of data stores in a set of data stores, and the inner loop for each data store—ceasing when the logical graph has been fully consumed. A query cost is determined for each of the permutations, and a physical plan is selected based on the query cost, for example minimizing the query cost, and the query is executed based on the selected physical plan. The cost may be determined based on a weighted combination of factors such as freshness of the data, latency, power consumption, environmental impact (e.g., carbon footprint), energy efficiency, and network burden.

Distribution of federated graph queries using schemas

The disclosed technology is generally directed to identification resolution in distributed knowledge graphs. In one example of the technology, a graph query is received. A metagraph includes identifier acceptance information associated with which identifiers among the identifiers are accepted by the data stores in the plurality of data stores and identifier export information associated with which identifiers among the identifiers are exported by the data stores in the plurality of data stores. The metagraph and the cost information are used to select, from among query paths, a query path that is capable of fulfilling the graph query based on minimizing a cost according to a cost metric. Upon determining that a failure has occurred in the selected query path, the metagraph and the cost information are used to change the selected query path. A response to the graph query is provided based on the selected query path.

Identification resolution in distributed knowledge graphs

Example solutions for executing a query in a declarative graph query language include receiving the query for data in a database and determining if one or both of i) a pattern in the query, and ii) the data in the database render the query suitable for being executed, at least in part, in parallel. If either condition indicates that the query is suitable for being executed, at least in part, in parallel, one or more fork operations and join operations are injected into a query plan, and the query is executed according to the query plan. Some examples further determine whether executing the query in parallel is computing resource-efficient, and only executes computing resource-efficient queries in parallel.

Instruction level parallelism in a declarative graph query language

Example solutions balance traversals and precomputations for connected data sets subject to access control. When a request for a commutative operation on a connected data set is received, a downwardly recursive process starts with the current hierarchy tier set to a highest hierarchy tier of the request. The process traverses the current hierarchy tier laterally. At each discovered node in the current hierarchy tier, a determination is made whether the requested operation is permitted to include a discovered node. If not, that node and any nodes hierarchically below it are omitted from the operation. That node and its subtree are skipped and the recursion halts for that branch. If, however, the requested operation is permitted to include the node, it is incorporated into the cumulative result and the recursion continues while there remains a lower hierarchy tier. Some examples leverage precomputed aggregates to avoid traversing branches having aggregates.

Computing resource conservation with balanced traversals and precomputations for connected data sets

Example solutions are disclosed for performing a distributed data query for a connected data set, such as a property graph or a relational database, distributed across a plurality of regions (e.g., different geographic regions) under data flow limitations. For a restrictive limitation, a first region stores a data entity that is subject to a data flow limitation, and a second region stores only a reference to the data entity (e.g., a pseudonymous reference). The query is executed in the first region, and at least a partial representation of the state is injected into the query for execution in the second region. The query locates the reference to the data entity in the second region, and the state of the query from the second region is returned. Query results from the plurality of regions are synthesized into a final result. Further solutions address prohibitive limitations when references are not permitted.

Distributed data query under data flow limitations

Techniques of reproduction of graph data during query time are disclosed herein. One example technique includes receiving, at a query processor, a query having a set of predicates to be evaluated on data in a graph. Upon receiving the query, the example technique includes evaluating the set of predicates based on data in the first or second partition of the graph and recording a sequence of query states of the first or second partition whose data is used to evaluate each of the set of predicates. Subsequently, the example technique includes constructing a set of snapshots of the data in the first or second partition based on the recorded query states and reevaluating the set of predicates on the constructed set of snapshots of the data in the first or second partition to troubleshoot the detected query error when the set of predicates were previously evaluated.

Reproduction of graph data during query evaluation

In various examples, there is provided methods performed by nodes in a cluster of nodes to establish a master clock at a new master node following a reconfiguration of the nodes included in the cluster, whereby the master clock is provided by an old master node prior to the reconfiguration, and synchronize a local clock of slave nodes to clock of the new master node. The new master node sends a message to the slave nodes instructing them to disable their respective local clocks, receives acknowledgements that the local clocks have been disabled, waits until a time at which all leases have expired for any nodes removed from the cluster, sets the value of its clock to be greater than a maximum value that could have been provided by the old master node at the time the leases expired and indicates to the other nodes to re-enable their local clocks.

Clock synchronization

In various examples, there is provided methods performed by nodes in a cluster of nodes for performing transactions comprising one or more read operations and/or one or more write operations. The node comprises a local clock which is synchronized with a master clock and maintains a measure of uncertainty indicating current minimum and maximum values of the master clock. The method to perform transactions involving read operations generates a read timestamp representing a point in time which is earlier than a current minimum value of the master clock. The method then reads the objects and determines, for each of them, whether a timestamp associated with that object is later than the read timestamp. If so, an error handling procedure is performed for that object.

Growing community of inventors

Tromso, Norway

Anders Tungeland Gjerdrum