Deepmind Technologies Limited

Gdm Holding LLC

Todd Andrew Hester

Nicolas Manfred Otto Heess

Mel Vecerik

Martin Riedmiller

Jonathan Karl Scholz

Thomas Lampe

Bilal Piot

Thomas Rothoerl

Olivier Claude Pietquin

Timothy Arthur Mann

Rae Chan Jeong

Wang Fumin

Abbas Abdolmaleki

Daniel J Mankowitz

Jost Tobias Springenberg

Nir Levine

A method of training an action selection neural network to perform a demonstrated task using a supervised learning technique. The action selection neural network is configured to receive demonstration data comprising actions to perform the task and rewards received for performing the actions. The action selection neural network has auxiliary prediction task neural networks on one or more of its intermediate outputs. The action selection policy neural network is trained using multiple combined losses, concurrently with the auxiliary prediction task neural networks.

Training action-selection neural networks from demonstrations using multiple losses

An off-policy reinforcement learning actor-critic neural network system configured to select actions from a continuous action space to be performed by an agent interacting with an environment to perform a task. An observation defines environment state data and reward data. The system has an actor neural network which learns a policy function mapping the state data to action data. A critic neural network learns an action-value (Q) function. A replay buffer stores tuples of the state data, the action data, the reward data and new state data. The replay buffer also includes demonstration transition data comprising a set of the tuples from a demonstration of the task within the environment. The neural network system is configured to train the actor neural network and the critic neural network off-policy using stored tuples from the replay buffer comprising tuples both from operation of the system and from the demonstration transition data.

Training action selection neural networks using apprenticeship

Growing community of inventors

Seattle, WA, United States of America

Todd Andrew Hester