Google Inc.

Andrew M Dai

Quoc V Le

Ramanathan V Guha

Vivek Raghunathan

Ramakrishnan Srikant

Kiran S Panesar

Pablo Bellver

Vineet Gupta

Aparna Chennapragada

Amay Nitin Champaneria

Frederick Peter Brewin

Mahesh Keralapura Manjunatha

Carolyn F Au

Karthik Lakshminarayanan

Carl Lischeske

Ravi Kolluri

Alvin Rishi Rajkomar

Kun Zhang

Gregory Sean Corrado

Yingwei Cui

Peter Junteng Liu

Jeffrey Adgate Dean

Kai Chen

Yi Zhang

James Wilson

Nissan Hajaj

Benjamin Irvine

Eyal Oren

James Wexler

Nan Du

Patrik Sundberg

David Ha

Hector Yee

Michaela Hardt

Xiaomi Sun

Xiaobing Liu

Alexander Mossin

Jacob Marcus

Jonas Beachey Kemp

Yonghui Wu

Balint Miklos

Tobias Kaufmann

Dengyong Zhou

Melvin Jose Johnson Premkumar

Thang Minh Luong

Orhan Firat

Itamar Gilad

Karol Piotr Kurach

Slav Petrov

Yanping Huang

Yuanzhong Xu

Ivo Krka

Aurko Roy

David Richard So

Maxim Krikun

Rohan Anil

Xavier Eduardo Garcia

Gavin Duggan

Kurt Litsch

Liam MacDermed

Gerardo Flores

Gaurav Mishra

Dmitry Lepikhin

Zhen Xu

Siamak Shakeri

Yong Cheng

Jonathan Hudson Clark

Erica Ann Moreira

Yujing Zhang

Zachary Alexander Nado

Kefan Xiao

Jin Young Sohn

Eric Jun Jie Ni

Vladimir Feinberg

Evan Witty

Alexandru Damian

Hoang Trieu Trinh

Yuan Xue

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform any one or more of a variety of machine learning tasks. For example, the neural network can be configured as a generative neural network, e.g., an autoregressive generative neural network.

Training of large neural networks

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes training a language model neural network, wherein the language model neural network comprises the one or more neural network layers and an output layer, and wherein training the language model neural network comprises determining pre-trained values of the parameters of the one or more neural network layers from initial values of the parameters of the one or more neural network layers; and training the neural network on a plurality of training documents to determine trained values of the parameters of the one or more neural network layers from the pre-trained values of the parameters of the one or more neural network layers.

Training a document classification neural network

A deep state space generative model is augmented with intervention prediction. The state space model provides a principled way to capture the interactions among observations, interventions, critical event occurrences, true states, and associated uncertainty. The state space model can include a discrete-time hazard rate model that provides flexible fitting of general survival time distributions. The state space model can output a joint prediction of event risk, observation and intervention trajectories based on patterns in temporal progressions, and correlations between past measurements and interventions.

Machine-learned state space model for joint forecasting

A system for predicting and summarizing medical events from electronic health records includes a computer memory storing aggregated electronic health records from a multitude of patients of diverse age, health conditions, and demographics including medications, laboratory values, diagnoses, vital signs, and medical notes. The aggregated electronic health records are converted into a single standardized data structure format and ordered arrangement per patient, e.g., into a chronological order. A computer (or computer system) executes one or more deep learning models trained on the aggregated health records to predict one or more future clinical events and summarize pertinent past medical events related to the predicted events on an input electronic health record of a patient having the standardized data structure format and ordered into a chronological order. An electronic device configured with a healthcare provider-facing interface displays the predicted one or more future clinical events and the pertinent past medical events of the patient.

System and method for predicting and summarizing medical events from electronic health records

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using recurrent neural networks. One of the systems includes a main recurrent neural network comprising one or more recurrent neural network layers and a respective hyper recurrent neural network corresponding to each of the one or more recurrent neural network layers, wherein each hyper recurrent neural network is configured to, at each of a plurality of time steps: process the layer input at the time step to the corresponding recurrent neural network layer, the current layer hidden state of the corresponding recurrent neural network layer, and a current hypernetwork hidden state of the hyper recurrent neural network to generate an updated hypernetwork hidden state.

Generating parameter values for recurrent neural networks

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a document classification neural network. One of the methods includes training an autoencoder neural network to autoencode input documents, wherein the autoencoder neural network comprises the one or more LSTM neural network layers and an autoencoder output layer, and wherein training the autoencoder neural network comprises determining pre-trained values of the parameters of the one or more LSTM neural network layers from initial values of the parameters of the one or more LSTM neural network layers; and training the document classification neural network on a plurality of training documents to determine trained values of the parameters of the one or more LSTM neural network layers from the pre-trained values of the parameters of the one or more LSTM neural network layers.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting future patient health using neural networks. One of the methods includes receiving electronic health record data for a patient; generating a respective observation embedding for each of the observations, comprising, for each clinical note: processing the sequence of tokens in the clinical note using a clinical note embedding LSTM to generate a respective token embedding for each of the tokens; and generating the observation embedding for the clinical note from the token embeddings; generating an embedded representation, comprising, for each time window: combining the observation embeddings of observations occurring during the time window to generate a patient record embedding; and processing the embedded representation of the electronic health record data using a prediction recurrent neural network to generate a neural network output that characterizes a future health status of the patient.

Processing clinical notes using recurrent neural networks

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for structuring and training a recurrent neural network. This describes a technique that improves the ability to capture long term dependencies in recurrent neural networks by adding an unsupervised auxiliary loss at one or more anchor points to the original objective. This auxiliary loss forces the network to either reconstruct previous events or predict next events in a sequence, making truncated backpropagation feasible for long sequences and also improving full backpropagation through time.

Learning longer-term dependencies in neural network using auxiliary losses

Methods and apparatus for providing travel-related information for a location to a user based on activity indications of the user that are related to the location. The location may be determined based on a set of one or more related activity indications and a travel-related score may be determined for the location that is indicative of likelihood that the user has interest in travelling to the location. The user may be provided the travel-related information for the location based on the travel-related score.

Automatic continued search

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating document vector representations. One of the methods includes obtaining a new document; selecting a plurality of new document word sets; and determining a vector representation for the new document using a trained neural network system, wherein the trained neural network system comprises: a document embedding layer and a classifier, and wherein determining the vector representation for the new document using the trained neural network system comprises iteratively providing each of the plurality of new document word sets to the trained neural network system to determine the vector representation for the new document using gradient descent.

Generating vector representations of documents

Growing community of inventors

San Francisco, CA, United States of America

Andrew M Dai