Deepmind Technologies Limited

Oath Inc.

Simon Osindero

Koray Kavukcuoglu

Joao Carreira

Viorica Patraucean

Andrew Zisserman

Alexander Vezhnevets

Oriol Vinyals

Timothy Paul Lillicrap

Alexander Benjamin Graves

Maxwell Elliot Jaderberg

Tom Schaul

Jack William Rae

Wojciech Czarnecki

Ioannis Kalantidis

William Clinton Dabney

Volodymyr Mnih

Bilal Piot

Georg Ostrovski

Bart Thomée

Clayton Ellis Mellina

Fengning Ding

Sergey Bartunov

Diana Luiza Borsa

David Szepesvari

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing associative memory. In one aspect a system comprises an associative memory neural network to process an input to generate an output that defines an energy corresponding to the input. A reading subsystem retrieves stored information from the associative memory neural network. The reading subsystem performs operations including receiving a given, i.e. query, input and retrieving a data element from the associative memory neural network that is associated with the given input. The retrieving is performed by iteratively adjusting the given input using the associative memory neural network.

Energy-based associative memory neural networks

A system that is configured to receive a sequence of task inputs and to perform a machine learning task is described. The system includes a reinforcement learning (RL) neural network and a task neural network. The RL neural network is configured to: generate, for each task input of the sequence of task inputs, a respective decision that determines whether to encode the task input or to skip the task input, and provide the respective decision of each task input to the task neural network. The task neural network is configured to: receive the sequence of task inputs, receive, from the RL neural network, for each task input of the sequence of task inputs, a respective decision that determines whether to encode the task input or to skip the task input, process each of the un-skipped task inputs in the sequence of task inputs to generate a respective accumulated feature for the un-skipped task input, wherein the respective accumulated feature characterizes features of the un-skipped task input and of previous un-skipped task inputs in the sequence, and generate a machine learning task output for the machine learning task based on the last accumulated feature generated for the last un-skipped task input in the sequence.

Reinforcement learning for active sequence processing

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes sampling a behavior modulation in accordance with a current probability distribution; for each of one or more time steps: processing an input comprising an observation characterizing a current state of the environment at the time step using an action selection neural network to generate a respective action score for each action in a set of possible actions that can be performed by the agent; modifying the action scores using the sampled behavior modulation; and selecting the action to be performed by the agent at the time step based on the modified action scores; determining a fitness measure corresponding to the sampled behavior modulation; and updating the current probability distribution over the set of possible behavior modulations using the fitness measure corresponding to the behavior modulation.

Modulating agent behavior to optimize learning progress

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for parallel processing of video frames using neural networks. One of the methods includes receiving a video sequence comprising a respective video frame at each of a plurality of time steps; and processing the video sequence using a video processing neural network to generate a video processing output for the video sequence, wherein the video processing neural network includes a sequence of network components, wherein the network components comprise a plurality of layer blocks each comprising one or more neural network layers, wherein each component is active for a respective subset of the plurality of time steps, and wherein each layer block is configured to, at each time step at which the layer block is active, receive an input generated at a previous time step and to process the input to generate a block output.

Parallel video processing systems

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network including a first subnetwork followed by a second subnetwork on training inputs by optimizing an objective function. In one aspect, a method includes processing a training input using the neural network to generate a training model output, including processing a subnetwork input for the training input using the first subnetwork to generate a subnetwork activation for the training input in accordance with current values of parameters of the first subnetwork, and providing the subnetwork activation as input to the second subnetwork; determining a synthetic gradient of the objective function for the first subnetwork by processing the subnetwork activation using a synthetic gradient model in accordance with current values of parameters of the synthetic gradient model; and updating the current values of the parameters of the first subnetwork using the synthetic gradient.

Training neural networks using synthetic gradients

Parallel video processing neural networks

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a system configured to select actions to be performed by an agent that interacts with an environment. The system comprises a manager neural network subsystem and a worker neural network subsystem. The manager subsystem is configured to, at each of the multiple time steps, generate a final goal vector for the time step. The worker subsystem is configured to, at each of multiple time steps, use the final goal vector generated by the manager subsystem to generate a respective action score for each action in a predetermined set of actions.

Action selection for reinforcement learning using a manager neural network that generates goal vectors defining agent objectives

One or more computing devices, systems, and/or methods for selective content insertion into areas of media objects are provided. For example, a media object (e.g., an image or video), is selected for composition with content, such as where a message, interactive content, a hyperlink, or other types of content is overlaid or embedded into the media object to create a composite media object. The content is added into an area of the media object that is selectively identified to reduce occlusion and/or improve visual cohesiveness between the content and the media object (e.g., added to an area with a similar or complimentary color, having an adequate size with spare amounts of visual features such as a soccer player, a ball, or other entity, etc.). In this way, the content may be add into the area of the media object to create a composite media object to provide to users.

Selective content insertion into areas of media objects

Action selection for reinforcement learning using neural networks

Growing community of inventors

London, United Kingdom

Simon Osindero