Deepmind Technologies Limited

Gdm Holding LLC

Misha Man Ray Denil

Joao Ferdinando Gomes De Freitas

Sergio Gomez Colmenarejo

Alexander Novikov

Koray Kavukcuoglu

Razvan Pascanu

Tom Schaul

Raia Thais Hadsell

Scott Ellison Reed

Mel Vecerik

David Budden

Hubert Josef Soyer

Jonathan Karl Scholz

Andrea Banino

Sudarshan Kumaran

Matthew William Hoffman

Yusuf Aytar

Laurent Sifre

David Benjamin Pfau

Marcin Andrychowicz

David Barker

Ksenia Konyushkova

Rostislav Goroshin

Oleg O Sushkov

Fabio Viola

Piotr Wojciech Mirowski

Andrew James Ballard

Serkan Cabi

Oriol Vinyals

Jack William Rae

Gabriel Barth-Maron

Ziyu Wang

Emilio Parisotto

Tom Erez

Rae Chan Jeong

Sergio Gomez

Konrad Zolna

Jacob Bruce

Ashley Deloris Edwards

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent to interact with an environment using an action selection neural network. In one aspect, a method comprises, at each time step in a sequence of time steps: generating a current representation of a state of a task being performed by the agent in the environment as of the current time step as a sequence of data elements; autoregressively generating a sequence of data elements representing a current action to be performed by the agent at the current time step; and after autoregressively generating the sequence of data elements representing the current action, causing the agent to perform the current action at the current time step.

Autoregressively generating sequences of data elements defining actions to be performed by an agent

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-driven robotic control. One of the methods includes maintaining robot experience data; obtaining annotation data; training, on the annotation data, a reward model; generating task-specific training data for the particular task, comprising, for each experience in a second subset of the experiences in the robot experience data: processing the observation in the experience using the trained reward model to generate a reward prediction, and associating the reward prediction with the experience; and training a policy neural network on the task-specific training data for the particular task, wherein the policy neural network is configured to receive a network input comprising an observation and to generate a policy output that defines a control policy for a robot performing the particular task.

Data-driven robot control

Methods, systems, and apparatus, including computer programs encoded on computer storage media for training machine learning models. One method includes obtaining a machine learning model, wherein the machine learning model comprises one or more model parameters, and the machine learning model is trained using gradient descent techniques to optimize an objective function; determining an update rule for the model parameters using a recurrent neural network (RNN); and applying a determined update rule for a final time step in a sequence of multiple time steps to the model parameters.

Training machine learning models by determining update rules using neural networks

Training machine learning models by determining update rules using recurrent neural networks

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a reinforcement learning system. In one aspect, a method of training an action selection policy neural network for use in selecting actions to be performed by an agent navigating through an environment to accomplish one or more goals comprises: receiving an observation image characterizing a current state of the environment; processing, using the action selection policy neural network, an input comprising the observation image to generate an action selection output; processing, using a geometry-prediction neural network, an intermediate output generated by the action selection policy neural network to predict a value of a feature of a geometry of the environment when in the current state; and backpropagating a gradient of a geometry-based auxiliary loss into the action selection policy neural network to determine a geometry-based auxiliary update for current values of the network parameters.

Environment navigation using reinforcement learning

Growing community of inventors

London, United Kingdom

Misha Man Ray Denil