Google Inc.

Dara Bahri

Donald Arthur Metzler, Jr

Mostafa Dehghani

Liu Yang

Da-Cheng Juan

Vinh Quoc Tran

Adam Joshua Fisch

Tal Schuster

Jai Prakash Gupta

Yi Tay

A system for performing a machine learning task on a network input is described. The system includes one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to implement (i) multiple sorting networks in which each sorting network is configured to sort vector blocks in a sequence of vector blocks to generate a sorted sequence of vector blocks; and (ii) a sorting attention neural network configured to perform the machine learning task on the input sequence by executing multiple sorting attention mechanisms using the sorting networks.

Sorting attention neural networks

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using auto-regressive decoder neural networks. In particular, during generation, adaptive early exiting is used to reduce the time required to generate the output sequence.

Efficient decoding of output sequences using adaptive early exiting

Growing community of inventors

Lafayette, CA, United States of America

Dara Bahri