Microsoft Technology Licensing, LLC

Google Inc.

Yuan Yu

Michael Isard

Jeffrey Adgate Dean

Mihai Budiu

Ulfar Erlingsson

Sanjay Ghemawat

Li Zhang

Thomas Lee Rodeheffer

Paul A Tucker

Dennis Craig Fetterly

Dharma Kiritkumar Shukla

Quetzalcoatl Bradley

Shireesh Kumar Thota

Madhan Gajendran

Pradeep Kumar Gunda

Zheng Zhang

Qifa Ke

Lintao Zhang

Thomas J Ball

Frank David McSherry

Andrew David Birrell

Yinglian Xie

Vijayan Prabhakaran

Carlo Aldo Curino

Rathijit Sen

Shuvendu K Lahiri

Subramaniam Venkatraman Krishnan

Raghunath Ramakrishnan

Kwanghyun Park

Konstantinos Karanasos

Zhengping Qian

Matteo Interlandi

JunFeng Yang

Xiuwei Chen

Markus Weimer

Li Zhuang

Manjunath Kudlur Venkatakrishna

Jingyue Wu

Fotios Psallidas

Gordon D Plotkin

Derek Gordon Murray

Ivan Popivanov

Doris Suiyi Xin

Karla Jean Saur

Paul Tucker

The description relates to executing an inference query relative to a database management system, such as a relational database management system. In one example a trained machine learning model can be stored within the database management system. An inference query can be received that applies the trained machine learning model on data local to the database management system. Analysis can be performed on the inference query and the trained machine learning model to generate a unified intermediate representation of the inference query and the trained model. Cross optimization can be performed on the unified intermediate representation. Based upon the cross-optimization, a first portion of the unified intermediate representation to be executed by a database engine of the database management system can be determined, and, a second portion of the unified intermediate representation to be executed by a machine learning runtime can be determined.

Query processing with machine learning

Systems and methods for processing loops in computational graphs representing machine learning models are disclosed. An example method begins with obtaining data representing a computational graph. Data identifying an allocation of the computational graph across devices is obtained. Additionally, one or more nodes in the computational graph that represent a respective control flow statement are identified. For each identified node, a structure of nodes and edges that represents an operation that provides a current state of recursion or iteration in the respective control flow statement is generated. This structure is inserted into the computational graph and the allocation of nodes to devices is modified to assign the structure to a device.

Processing loops in computational graphs

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for receiving a request from a client to process a computational graph; obtaining data representing the computational graph, the computational graph comprising a plurality of nodes and directed edges, wherein each node represents a respective operation, wherein each directed edge connects a respective first node to a respective second node that represents an operation that receives, as input, an output of an operation represented by the respective first node; identifying a plurality of available devices for performing the requested operation; partitioning the computational graph into a plurality of subgraphs, each subgraph comprising one or more nodes in the computational graph; and assigning, for each subgraph, the operations represented by the one or more nodes in the subgraph to a respective available device in the plurality of available devices for operation.

Processing computational graphs

Systems and Methods for training a neural network represented as a computational graph are disclosed. An example method begins with obtaining data representing a computational graph. The computational graph is then augmented to generate a training computational graph for training the neural network using a machine learning training algorithm that includes computing a gradient of an objective function with respect to each of the parameters of the neural network. Augmenting the computational graph includes inserting a plurality of gradient nodes and training edges into the computational graph to generate a backward path through the computational graph that represents operations for computing the gradients of the objective function with respect to the parameters of the neural network. The neural network is trained using the machine learning training algorithm by executing the training computational graph.

Training neural networks represented as computational graphs

General-purpose distributed data-parallel computing using a high-level language is disclosed. Data parallel portions of a sequential program that is written by a developer in a high-level language are automatically translated into a distributed execution plan. The distributed execution plan is then executed on large compute clusters. Thus, the developer is allowed to write the program using familiar programming constructs in the high level language. Moreover, developers without experience with distributed compute systems are able to take advantage of such systems.

General purpose distributed data parallel computing using a high level language

Query models for document sets (such as XML documents or records in a relational database) typically involve a schema defining the structure of the documents. However, rigidly defined schemas often raise difficulties with document validation with even inconsequential structural variations. Additionally, queries developed against schema-constrained documents are often sensitive to structural details and variations that are not inconsequential to the query, resulting in inaccurate results and development complications, and that may break upon schema changes. Instead, query models for hierarchically structured documents that enable 'twig' queries specifying only the structural details of document nodes that are relevant to the query (e.g., students in a student database having a sibling named 'Lee' and a teacher named &#x201c;Smith,&#x201d; irrespective of unrelated structural details of the document). Such &#x201c;twig&#x201d; query models may enable a more natural query development, and continued accuracy of queries in the event of unrelated schema variations and changes.

Scalable, schemaless document query model

A data partitioning plan is automatically generated that&#x2014;given a data-parallel program and a large input dataset, and without having to first run the program on the input dataset&#x2014;substantially optimizes performance of the distributed execution system that explicitly measures and infers various properties of both data and computation to perform cost estimation and optimization. Estimation may comprise inferring the cost of a candidate data partitioning plan, and optimization may comprise generating an optimal partitioning plan based on the estimated costs of computation and input/output.

Optimizing data partitioning for data-parallel computing

A document index is generated from a set of documents and is used to identify documents that match one or more queries. A tree is generated for each document with a node corresponding to each object of the document. The nodes of the generated trees are merged or combined to generate the document index, which is itself a tree. In addition, an inverted index is generated for each node of the index that identifies the tree(s) that the node originated from. When a query is received, the query is first executed against the document index tree: during the execution, proper set operations are applied to the inverted indices associated with the nodes matched by the query. The resulted set identifies the documents that may match the query. The query is then executed on the identified documents.

Query and index over documents

Growing community of inventors

Cupertino, CA, United States of America

Yuan Yu