Google Inc.

Gaurav Mishra

Noam M Shazeer

Yonghui Wu

Andrew M Dai

Melvin Jose Johnson Premkumar

Orhan Firat

Slav Petrov

Adam Roberts

Yanping Huang

Yuanzhong Xu

Maxim Krikun

Nan Du

Aurko Roy

David Richard So

Rohan Anil

Maarten Paul Bosma

Xavier Eduardo Garcia

Dmitry Lepikhin

Siamak Shakeri

Yong Cheng

Jonathan Hudson Clark

Erica Ann Moreira

Yujing Zhang

Zachary Alexander Nado

Kefan Xiao

Jin Young Sohn

Eric Jun Jie Ni

Vladimir Feinberg

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform any one or more of a variety of machine learning tasks. For example, the neural network can be configured as a generative neural network, e.g., an autoregressive generative neural network.

Training of large neural networks

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model using a deterministic data pipeline. One of the methods may include receiving a first request to generate a deterministic training dataset: transforming raw training examples obtained from the raw data source into pre-processed training examples; assigning a unique index to each pre-processed training example; and caching the pre-processed training examples into the cache directory specified in the received first request; receiving a second request to use the deterministic training dataset to train a machine learning model, the second request specifying a start index; and in response to receiving the second request: reading, from the cache directory, the pre-processed training examples that have indices beginning from the start index; and providing the read training examples in an order of the assigned indices for use in training the machine learning model.

Deterministic training of machine learning models

Growing community of inventors

Sunnyvale, CA, United States of America

Gaurav Mishra