Google Inc.

Microsoft Technology Licensing, LLC

Sudip Roy

Azalia Mirhoseini

Qiumin Xu

Yanqi Zhou

Amirali Abdolrashidi

Daniel Lin-Kit Wong

Jeffrey Adgate Dean

Sanjay Ghemawat

Yonghui Wu

Ruoming Pang

Michael Isard

Chandramohan Amyangot Thekkath

Paul Ronald Barham

Ryan Sepassi

Aakanksha Chowdhery

Laurent El Shafey

Parker Edward Schuh

Hyeontaek Lim

Daniel William Hurt

Brennan Saeta

Chao Ma

Alon Yitzchak Halevy

James Laudon

Christopher Olston

Anna Darling Goldie

Philip Korn

Shen Wang

Hanxiao Liu

Steven Euijong Whang

Igor Dvorkin

Manish Kumar

Phitchaya Mangpo Phothilimthana

Neoklis Polyzotis

Natalya Fridman Noy

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for compiler optimizations using a compiler optimization network. One of the methods includes receiving an input program, wherein the input program defines a graph of operation modules, wherein each node in the graph is a respective operation module, and each edge between nodes in the graph represents one operation module receiving the output generated by another operation module. The input program is processed by a compiler optimization network comprising a graph-embedding network that is configured to encode operation features and operation dependencies of the operation modules of the input program into a graph embedding representation and a policy network that is configured to generate an optimization action for each of one or more nodes encoded in the graph embedding representation. The compiler optimization network generates an output optimization plan comprising one or more optimization actions for the input program.

Learned graph optimizations for compilers

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for optimizing the execution of the operations of a neural network. One of the methods includes obtaining data representing a graph characterizing a plurality of operations of a neural network, wherein each node of the graph characterizes an operation of the neural network and each edge of the graph characterizes data dependency between the operations; processing the data representing the graph using a graph embedding neural network to generate an embedding of the graph; and processing the embedding of the graph using a policy neural network to generate a task output, wherein the task output comprises, for each of the plurality of operations of the neural network, a respective decision for a particular optimization task.

Computational graph optimization

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing machine learning workloads, e.g., computations for training a neural network or computing an inference using a neural network, across multiple hardware accelerators. One of the systems comprises a plurality of accelerator islands, each hardware accelerator island comprising a respective plurality of hardware devices that include a plurality of hardware accelerators and a corresponding host for each of the plurality of hardware accelerators; and a respective scheduler for each of the accelerator islands that is configured to schedule workloads across the plurality of accelerators and corresponding hosts in the accelerator island, wherein the system is configured to: receive data representing a machine learning workload; and assign a respective portion of the machine learning workload to each of the plurality of accelerator islands for scheduling by the respective scheduler for the accelerator island.

Asynchronous distributed data flow for machine learning workloads

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a catalog for multiple datasets, the method comprising accessing multiple extant data sets, the extant data sets including data sets that are independently generated and structurally dissimilar; organizing the data sets into collections, each data set in each collection belonging to the collection based on collection data associated with the data set; for each collection of data sets: determining, from a subset of the data sets that belong to the collection, metadata that describe the data sets that belong to the collection, wherein the metadata does not include the collection data, and attributing, to other data sets in the collection, the metadata determined from the subset of data sets; and generating, from the collections of data sets and the determined metadata, a catalog for the multiple datasets.

Post-hoc management of datasets

The described implementations relate to tunable predicate discovery. One implementation is manifest as a method for obtaining a data set and determining anomaly scores for anomalies of an attribute of interest in the data set. The method can also generate a ranked list of predicates based on the anomaly scores and cause at least one of the predicates of the ranked list to be presented.

Performance anomaly diagnosis

Growing community of inventors

San Jose, CA, United States of America

Sudip Roy