Deepmind Technologies Limited

Olivier Claude Pietquin

Bilal Piot

Nicolas Manfred Otto Heess

Charles Blundell

Mel Vecerik

Remi Munos

Martin Riedmiller

Jonathan Karl Scholz

Thomas Lampe

Jacob Lee Menick

Meire Fortunato

Thomas Rothoerl

Mohammad Gheshlaghi Azar

Todd Andrew Hester

Wang Fumin

Volodymyr Mnih

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting an action to be performed by a reinforcement learning agent. The method includes obtaining an observation characterizing a current state of an environment. For each layer parameter of each noisy layer of a neural network, a respective noise value is determined. For each layer parameter of each noisy layer, a noisy current value for the layer parameter is determined from a current value of the layer parameter, a current value of a corresponding noise parameter, and the noise value. A network input including the observation is processed using the neural network in accordance with the noisy current values to generate a network output for the network input. An action is selected from a set of possible actions to be performed by the agent in response to the observation using the network output.

Noisy neural network layers with noise parameters

An off-policy reinforcement learning actor-critic neural network system configured to select actions from a continuous action space to be performed by an agent interacting with an environment to perform a task. An observation defines environment state data and reward data. The system has an actor neural network which learns a policy function mapping the state data to action data. A critic neural network learns an action-value (Q) function. A replay buffer stores tuples of the state data, the action data, the reward data and new state data. The replay buffer also includes demonstration transition data comprising a set of the tuples from a demonstration of the task within the environment. The neural network system is configured to train the actor neural network and the critic neural network off-policy using stored tuples from the replay buffer comprising tuples both from operation of the system and from the demonstration transition data.

Training action selection neural networks using apprenticeship

Growing community of inventors

Lille, France

Olivier Claude Pietquin