Deepmind Technologies Limited

Thomas Rothoerl

Nicolas Manfred Otto Heess

Mel Vecerik

Razvan Pascanu

Raia Thais Hadsell

Andrei-Alexandru Rusu

Martin Riedmiller

Todd Andrew Hester

Jonathan Karl Scholz

Thomas Lampe

Bilal Piot

Olivier Claude Pietquin

Wang Fumin

Matej Vecerik

Olivier Pietquin

An off-policy reinforcement learning actor-critic neural network system configured to select actions from a continuous action space to be performed by an agent interacting with an environment to perform a task. An observation defines environment state data and reward data. The system has an actor neural network which learns a policy function mapping the state data to action data. A critic neural network learns an action-value (Q) function. A replay buffer stores tuples of the state data, the action data, the reward data and new state data. The replay buffer also includes demonstration transition data comprising a set of the tuples from a demonstration of the task within the environment. The neural network system is configured to train the actor neural network and the critic neural network off-policy using stored tuples from the replay buffer comprising tuples both from operation of the system and from the demonstration transition data.

Training action selection neural networks using apprenticeship

A system includes a neural network system implemented by one or more computers. The neural network system is configured to receive an observation characterizing a current state of a real-world environment being interacted with by a robotic agent to perform a robotic task and to process the observation to generate a policy output that defines an action to be performed by the robotic agent in response to the observation. The neural network system includes: (i) a sequence of deep neural networks (DNNs), in which the sequence of DNNs includes a simulation-trained DNN that has been trained on interactions of a simulated version of the robotic agent with a simulated version of the real-world environment to perform a simulated version of the robotic task, and (ii) a first robot-trained DNN that is configured to receive the observation and to process the observation to generate the policy output.

Neural networks for selecting actions to be performed by a robotic agent

Growing community of inventors

London, United Kingdom

Thomas Rothoerl