Google Inc.

Compaq Computer Corporation, Inc.

Digital Equipment Corporation

Alta Vista Company

Overture Services, Inc.

Jeffrey Adgate Dean

Sanjay Ghemawat

Gregory Sean Corrado

Georges R Harik

Kai Chen

Simon Tong

Paul Haahr

Olcan Sercinoglu

Carl Alan Waldspurger

Paul Thomas Buchheit

Monika Hildegard Henzinger

William Edward Weihl

Karl Pfleger

Steve Lawrence

James E Hicks, Jr

Quoc V Le

Krishna Asur Bharat

Benedict A Gomes

Yasushi Saito

Azalia Mirhoseini

Samy Bengio

Anurag Acharya

Andrew Fikes

George Z Chrysos

Chian-Min Richard Ho

Peng Xu

Ilya Sutskever

Thorsten Brants

Franz Josef Och

Yingwei Cui

Matthieu Devin

Michael Burrows

Sean Quinlan

Narayanan Shivakumar

Yuan Yu

Noam M Shazeer

Jay Michael Ponte

Deepak Jindal

Alexander Mark Franz

Jonathon Shlens

Jeffrey Chin

Sean Matthew Dorward

Gautham K Thambidorai

Daniel Rosart

Oriol Vinyals

Kun Zhang

Geoffrey Everest Hinton

Alexandre A Verstak

Fritz Schneider

Alexis J Battle

Amitabh Singhal

Daniel Wesley Dulitz

Yoram Singer

Corin Anderson

James Laudon

Emre Tuncer

Eric Russell Fredricksen

Andrei Z Broder

Andrea Lynn Frome

Edward J McLellan

Sachin Jain

Boris Mazniker

Vijay Vasudevan

Daniel Lawrence Leibholz

Niels Provos

Mohammad Norouzi

Paul A Tucker

Naveen Kumar

Yi Zhang

Yonghui Wu

Scott Davies

James Wilson

Alexander Lloyd

Michael Curtiss

Kourosh Gharachorloo

Ruoming Pang

Michael Isard

Chandramohan Amyangot Thekkath

Wilson Cheng-Yi Hsieh

Michael Schmitt

Bahman Rabii

Lawrence E Page

Andrew M Dai

Samuel Bengio

Ashok Chhabedia Popat

Michael James Boyer Epstein

Nissan Hajaj

Joseph Kieran O'Sullivan

Xiaodan Song

Alvin Rishi Rajkomar

Sebastian Kanthak

Nitin Kumar Gupta

Andrew William Senior

Vincent Olivier Vanhoucke

Ciprian Ioan Chelba

Luiz Andre Barroso

Anwis Das

Patrick An Phu Nguyen

Georg Heigold

Emanuel Taropa

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing numeric representations of words. One of the methods includes obtaining a set of training data, wherein the set of training data comprises sequences of words; training a classifier and an embedding function on the set of training data, wherein training the embedding function comprises obtained trained values of the embedding function parameters; processing each word in the vocabulary using the embedding function in accordance with the trained values of the embedding function parameters to generate a respective numerical representation of each word in the vocabulary in the high-dimensional space; and associating each word in the vocabulary with the respective numeric representation of the word in the high-dimensional space.

Computing numeric representations of words in a high-dimensional space

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network system to perform multiple machine learning tasks.

Training neural network systems to perform multiple machine learning tasks

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip placement. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip placement, comprising placing a respective macro node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the macro node to be placed at the time step to a position from the plurality of positions using the score distribution.

Generating integrated circuit placements using neural networks

Systems and methods for processing loops in computational graphs representing machine learning models are disclosed. An example method begins with obtaining data representing a computational graph. Data identifying an allocation of the computational graph across devices is obtained. Additionally, one or more nodes in the computational graph that represent a respective control flow statement are identified. For each identified node, a structure of nodes and edges that represents an operation that provides a current state of recursion or iteration in the respective control flow statement is generated. This structure is inserted into the computational graph and the allocation of nodes to devices is modified to assign the structure to a device.

Processing loops in computational graphs

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing machine learning workloads, e.g., computations for training a neural network or computing an inference using a neural network, across multiple hardware accelerators. One of the systems comprises a plurality of accelerator islands, each hardware accelerator island comprising a respective plurality of hardware devices that include a plurality of hardware accelerators and a corresponding host for each of the plurality of hardware accelerators; and a respective scheduler for each of the accelerator islands that is configured to schedule workloads across the plurality of accelerators and corresponding hosts in the accelerator island, wherein the system is configured to: receive data representing a machine learning workload; and assign a respective portion of the machine learning workload to each of the plurality of accelerator islands for scheduling by the respective scheduler for the accelerator island.

Asynchronous distributed data flow for machine learning workloads

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a computer chip floorplan. One of the methods includes obtaining netlist data for a computer chip; and generating a computer chip floorplan, comprising placing a respective node at each time step in a sequence comprising a plurality of time steps, the placing comprising, for each time step: generating an input representation for the time step; processing the input representation using a node placement neural network having a plurality of network parameters, wherein the node placement neural network is configured to process the input representation in accordance with current values of the network parameters to generate a score distribution over a plurality of positions on the surface of the computer chip; and assigning the node to be placed at the time step to a position from the plurality of positions using the score distribution.

Generating integrated circuit floorplans using neural networks

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting likelihoods of conditions being satisfied using recurrent neural networks. One of the systems is configured to process a temporal sequence comprising a respective input at each of a plurality of time steps and comprises: one or more recurrent neural network layers; one or more logistic regression nodes, wherein each of the logistic regression nodes corresponds to a respective condition from a predetermined set of conditions, and wherein each of the logistic regression nodes is configured to, for each of the plurality of time steps: receive the network internal state for the time step; and process the network internal state for the time step in accordance with current values of a set of parameters of the logistic regression node to generate a future condition score for the corresponding condition for the time step.

Predicting likelihoods of conditions being satisfied using neural networks

A method for deleting obsolete files from a file system is provided. The method includes receiving a request to delete a reference to a first target file of a plurality of target files stored in a file system, the first target file having a first target file name. A first reference file whose file name includes the first target file name is identified. The first reference file is deleted from the file system. The method further includes determining whether the file system includes at least one reference file, distinct from the first reference file, whose file name includes the first target file name. In accordance with a determination that the file system does not include the at least one reference file, the first target file is deleted from the file system.

Method and system for deleting obsolete files from a file system

The present disclosure provides systems and methods that generate new architectures for artificial neural networks based on connectomics data that describes connections between biological neurons of a biological organism. In particular, in some implementations, a computing system can identify one or more new artificial neural network architectures by performing a neural architecture search over a search space that is constrained based at least in part on the connectomics data.

Connectomics-based neural architecture search

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classifying data objects. One of the methods includes obtaining data that associates each term in a vocabulary of terms with a respective high-dimensional representation of the term; obtaining classification data for a data object, wherein the classification data includes a respective score for each of a plurality of categories, and wherein each of the categories is associated with a respective category label; computing an aggregate high-dimensional representation for the data object from high-dimensional representations for the category labels associated with the categories and the respective scores; identifying a first term in the vocabulary of terms having a high-dimensional representation that is closest to the aggregate high-dimensional representation; and selecting the first term as a category label for the data object.

Classifying data objects

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using embedded function with a deep network. One of the methods includes receiving an input comprising a plurality of features, wherein each of the features is of a different feature type; processing each of the features using a respective embedding function to generate one or more numeric values, wherein each of the embedding functions operates independently of each other embedding function, and wherein each of the embedding functions is used for features of a respective feature type; processing the numeric values using a deep network to generate a first alternative representation of the input, wherein the deep network is a machine learning model composed of a plurality of levels of non-linear operations; and processing the first alternative representation of the input using a logistic regression classifier to predict a label for the input.

Using embedding functions with a deep network

A system for predicting and summarizing medical events from electronic health records includes a computer memory storing aggregated electronic health records from a multitude of patients of diverse age, health conditions, and demographics including medications, laboratory values, diagnoses, vital signs, and medical notes. The aggregated electronic health records are converted into a single standardized data structure format and ordered arrangement per patient, e.g., into a chronological order. A computer (or computer system) executes one or more deep learning models trained on the aggregated health records to predict one or more future clinical events and summarize pertinent past medical events related to the predicted events on an input electronic health record of a patient having the standardized data structure format and ordered into a chronological order. An electronic device configured with a healthcare provider-facing interface displays the predicted one or more future clinical events and the pertinent past medical events of the patient.

System and method for predicting and summarizing medical events from electronic health records

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a distilled machine learning model. One of the methods includes training a cumbersome machine learning model, wherein the cumbersome machine learning model is configured to receive an input and generate a respective score for each of a plurality of classes; and training a distilled machine learning model on a plurality of training inputs, wherein the distilled machine learning model is also configured to receive inputs and generate scores for the plurality of classes, comprising: processing each training input using the cumbersome machine learning model to generate a cumbersome target soft output for the training input; and training the distilled machine learning model to, for each of the training inputs, generate a soft output that matches the cumbersome target soft output for the training input.

Training distilled machine learning models

Growing community of inventors

Palo Alto, CA, United States of America

Jeffrey Adgate Dean