Google Inc.

Alexey Dosovitskiy

Neil Matthew Tinmouth Houlsby

Dirk Weissenborn

Matthias Johannes Lorenz Minderer

Jonathan Tilton Barron

Jakob D Uszkoreit

Austin Charles Stone

Seyed Mohammad Mehdi Sajjadi

Georg Heigold

Alexander Kolesnikov

Lucas Klaus Beyer

Xiaohua Zhai

Daniel Christopher Duckworth

Thomas Unterthiner

Mostafa Dehghani

Sylvain Gelly

Alexey Alexeevich Gritsenko

Ricardo Martin-Brualla

Daniel M Keysers

Ricardo Martin Brualla

Sara Sabour Rouh Aghdam

Noha Waheed Ahmed Radwan

Mario Lucic

Gamaleldin Elsayed

Yin Ching Jessica Yung

Andreas Peter Steiner

Noha Radwan

Rico Jonschkowski

Thomas Kipf

Aravindh Mahendran

Klaus Greff

Ilya Tolstikhin

A method includes obtaining first feature vectors and second feature vectors representing contents of a first and second image frame, respectively, of an input video. The method may also include generating, based on the first feature vectors, first slot vectors, where each slot vector represents attributes of a corresponding entity as represented in the first image frame, and generating, based on the first slot vectors, predicted slot vectors including a corresponding predicted slot vector that represents a transition of the attributes of the corresponding entity from the first to the second image frame. The method may additionally include generating, based on the predicted slot vectors and the second feature vectors, second slot vectors including a corresponding slot vector that represents the attributes of the corresponding entity as represented in the second image frame, and determining an output based on the predicted slot vectors or the second slot vectors.

Conditional object-centric learning with slot attention for video and other sequential data

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images using mixer neural networks. One of the methods includes obtaining one or more images comprising a plurality of pixels; determining, for each image of the one or more images, a plurality of image patches of the image, wherein each image patch comprises a different subset of the pixels of the image; processing, for each image of the one or more images, the corresponding plurality of image patches to generate an input sequence comprising a respective input element at each of a plurality of input positions, wherein a plurality of the input elements correspond to respective different image patches; and processing the input sequences using a neural network to generate a network output that characterizes the one or more images, wherein the neural network comprises one or more mixer neural network layers.

Multi-layer perceptron-based computer vision neural networks

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for object detection. In one aspect, a method comprises: obtaining: (i) an image, and (ii) a set of one or more query embeddings, wherein each query embedding represents a respective category of object; processing the image and the set of query embeddings using an object detection neural network to generate object detection data for the image, comprising: processing the image using an image encoding subnetwork of the object detection neural network to generate a set of object embeddings; processing each object embedding using a localization subnetwork to generate localization data defining a corresponding region of the image; and processing: (i) the set of object embeddings, and (ii) the set of query embeddings, using a classification subnetwork to generate, for each object embedding, a respective classification score distribution over the set of query embeddings.

Open-vocabulary object detection in images

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images using self-attention based neural networks. One of the methods includes obtaining one or more images comprising a plurality of pixels; determining, for each image of the one or more images, a plurality of image patches of the image, wherein each image patch comprises a different subset of the pixels of the image; processing, for each image of the one or more images, the corresponding plurality of image patches to generate an input sequence comprising a respective input element at each of a plurality of input positions, wherein a plurality of the input elements correspond to respective different image patches; and processing the input sequences using a neural network to generate a network output that characterizes the one or more images, wherein the neural network comprises one or more self-attention neural network layers.

Processing images using self-attention based neural networks

Provided are systems and methods for synthesizing novel views of complex scenes (e.g., outdoor scenes). In some implementations, the systems and methods can include or use machine-learned models that are capable of learning from unstructured and/or unconstrained collections of imagery such as, for example, 'in the wild' photographs. In particular, example implementations of the present disclosure can learn a volumetric scene density and radiance represented by a machine-learned model such as one or more multilayer perceptrons (MLPs).

View synthesis robust to unconstrained image data

Growing community of inventors

Berlin, Germany

Alexey Dosovitskiy