Google Inc.

Aravindh Mahendran

Alexey Dosovitskiy

Jakob D Uszkoreit

Dirk Weissenborn

Sara Sabour Rouh Aghdam

Georg Heigold

Austin Charles Stone

Thomas Unterthiner

Gamaleldin Elsayed

Thomas Kipf

Klaus Greff

Jean-Baptiste Cordonnier

Rico Jonschkowski

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to process images. One of the methods includes obtaining a training image; processing the training image using a first subnetwork to generate, for each of a plurality of first image patches of the training image, a relevance score; generating, using the relevance scores, one or more second image patches of the training image by performing one or more differentiable operations on the relevance scores; processing the one or more second image patches using a second subnetwork to generate a prediction about the training image; determining an error of the training network output; and generating a parameter update for the first subnetwork, comprising backpropagating gradients determined according to the error of the training network output through i) the second subnetwork, ii) the one or more differentiable operations, and iii) the first subnetwork.

End-to-end training of neural networks for image processing

A method includes obtaining first feature vectors and second feature vectors representing contents of a first and second image frame, respectively, of an input video. The method may also include generating, based on the first feature vectors, first slot vectors, where each slot vector represents attributes of a corresponding entity as represented in the first image frame, and generating, based on the first slot vectors, predicted slot vectors including a corresponding predicted slot vector that represents a transition of the attributes of the corresponding entity from the first to the second image frame. The method may additionally include generating, based on the predicted slot vectors and the second feature vectors, second slot vectors including a corresponding slot vector that represents the attributes of the corresponding entity as represented in the second image frame, and determining an output based on the predicted slot vectors or the second slot vectors.

Conditional object-centric learning with slot attention for video and other sequential data

Growing community of inventors

Berlin, Germany

Aravindh Mahendran