Deepmind Technologies Limited

David Budden

Gabriel Barth-Maron

Scott Ellison Reed

Sergio Gomez Colmenarejo

Matthew William Hoffman

Yusuf Aytar

Alexander Novikov

Joao Ferdinando Gomes De Freitas

Mel Vecerik

Ziyu Wang

Jonathan Karl Scholz

Misha Man Ray Denil

John Quan

David Barker

Daniel George Horgan

Serkan Cabi

Ksenia Konyushkova

Oleg O Sushkov

Oriol Vinyals

Aaron Gerard Antonius Van Den Oord

Maxwell Elliot Jaderberg

Timothy James Alexander Harley

Chenjie Gu

Valentin Clement Dalibard

Sagi Perel

Rae Chan Jeong

Ang Li

Pramod Gupta

Ola Spyra

Tobias Pfaff

Tom Paine

Konrad Zolna

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for data-driven robotic control. One of the methods includes maintaining robot experience data; obtaining annotation data; training, on the annotation data, a reward model; generating task-specific training data for the particular task, comprising, for each experience in a second subset of the experiences in the robot experience data: processing the observation in the experience using the trained reward model to generate a reward prediction, and associating the reward prediction with the experience; and training a policy neural network on the task-specific training data for the particular task, wherein the policy neural network is configured to receive a network input comprising an observation and to generate a policy output that defines a control policy for a robot performing the particular task.

Data-driven robot control

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network used to select actions to be performed by an agent interacting with an environment. One of the systems includes (i) a plurality of actor computing units, in which each of the actor computing units is configured to maintain a respective replica of the action selection neural network and to perform a plurality of actor operations, and (ii) one or more learner computing units, in which each of the one or more learner computing units is configured to perform a plurality of learner operations.

Reinforcement learning using distributed prioritized replay

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection neural network that is used to select actions to be performed by a reinforcement learning agent interacting with an environment. In particular, the actions are selected from a continuous action space and the system trains the action selection neural network jointly with a distribution Q network that is used to update the parameters of the action selection neural network.

Distributional reinforcement learning for continuous control tasks

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a machine learning model. A method includes: maintaining a plurality of training sessions; assigning, to each worker of one or more workers, a respective training session of the plurality of training sessions; repeatedly performing operations until meeting one or more termination criteria, the operations comprising: receiving an updated training session from a respective worker of the one or more workers, selecting a second training session, selecting, based on comparing the updated training session and the second training session using a fitness evaluation function, either the updated training session or the second training session as a parent training session, generating a child training session from the selected parent training session, and assigning the child training session to an available worker, and selecting a candidate model to be a trained model for the machine learning model.

Population-based training of machine learning models

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection policy neural network, wherein the action selection policy neural network is configured to process an observation characterizing a state of an environment to generate an action selection policy output, wherein the action selection policy output is used to select an action to be performed by an agent interacting with an environment. In one aspect, a method comprises: obtaining an observation characterizing a state of the environment subsequent to the agent performing a selected action; generating a latent representation of the observation; processing the latent representation of the observation using a discriminator neural network to generate an imitation score; determining a reward from the imitation score; and adjusting the current values of the action selection policy neural network parameters based on the reward using a reinforcement learning training technique.

Action selection neural network training using imitation learning in latent space

Growing community of inventors

London, United Kingdom

David Budden