Google Inc.

Kumar Avinava Dubey

Manzil Zaheer

Amr Ahmed

Santiago Ontañón

Guru Guruganesh

Joshua Timothy Ainslie

Philip Pham

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing network inputs using an attention neural network that has one or more sparse attention sub-layers. Each sparse attention sub-layer is configured to apply a sparse attention mechanism that attends differently for input positions that are in a first proper subset of the input positions in the input to the sub-layer than for positions that are not in the first proper subset.

Growing community of inventors

Palo Alto, CA, United States of America

Kumar Avinava Dubey