Deepmind Technologies Limited

Gdm Holding LLC

Emilio Parisotto

Jack William Rae

Razvan Pascanu

Maxwell Elliot Jaderberg

Siddhant Jayakumar

Hasuk Song

Oriol Vinyals

Joao Ferdinando Gomes De Freitas

Gabriel Barth-Maron

Misha Man Ray Denil

Alexander Novikov

Tom Erez

Ashley Deloris Edwards

Caglar Gulcehre

Scott Ellison Reed

Sergio Gomez

Konrad Zolna

Jacob Bruce

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent to interact with an environment using an action selection neural network. In one aspect, a method comprises, at each time step in a sequence of time steps: generating a current representation of a state of a task being performed by the agent in the environment as of the current time step as a sequence of data elements; autoregressively generating a sequence of data elements representing a current action to be performed by the agent at the current time step; and after autoregressively generating the sequence of data elements representing the current action, causing the agent to perform the current action at the current time step.

Autoregressively generating sequences of data elements defining actions to be performed by an agent

A system including an attention neural network that is configured to receive an input sequence and to process the input sequence to generate an output is described. The attention neural network includes: an attention block configured to receive a query input, a key input, and a value input that are derived from an attention block input. The attention block includes an attention neural network layer configured to: receive an attention layer input derived from the query input, the key input, and the value input, and apply an attention mechanism to the query input, the key input, and the value input to generate an attention layer output for the attention neural network layer; and a gating neural network layer configured to apply a gating mechanism to the attention block input and the attention layer output of the attention neural network layer to generate a gated attention output.

Gated attention neural networks

Growing community of inventors

London, United Kingdom

Emilio Parisotto