Google Inc.

Gerhard Florian Schroff

Dmitry Kalenichenko

Hartwig Adam

James William Philbin

Liang-Chieh Chen

Georgios Papandreou

Keren Ye

Jonathon Shlens

Barret Zoph

Peng Wang

Yukun Zhu

Wenze Hu

Alexander Hermans

Maxwell Donald Emmet Collins

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an object embedding system. In one aspect, a method comprises providing selected images as input to the object embedding system and generating corresponding embeddings, wherein the object embedding system comprises a thumbnailing neural network and an embedding neural network. The method further comprises backpropagating gradients based on a loss function to reduce the distance between embeddings for same instances of objects, and to increase the distance between embeddings for different instances of objects.

Generating object embeddings from images

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for instance segmentation. In one aspect, a system generates: (i) data identifying one or more regions of the image, wherein an object is depicted in each region, (ii) for each region, a predicted type of object that is depicted in the region, and (iii) feature channels comprising a plurality of semantic channels and one or more direction channels. The system generates a region descriptor for each of the one or more regions, and provides the region descriptor for each of the one or more regions to a segmentation neural network that processes a region descriptor for a region to generate a predicted segmentation of the predicted type of object depicted in the region.

Instance segmentation

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining neural network architectures. One of the methods includes obtaining training data for a dense image prediction task; and determining an architecture for a neural network configured to perform the dense image prediction task, comprising: searching a space of candidate architectures to identify one or more best performing architectures using the training data, wherein each candidate architecture in the space of candidate architectures comprises (i) the same first neural network backbone that is configured to receive an input image and to process the input image to generate a plurality of feature maps and (ii) a different dense prediction cell configured to process the plurality of feature maps and to generate an output for the dense image prediction task; and determining the architecture for the neural network based on the best performing candidate architectures.

Neural architecture search for dense image prediction tasks

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating numeric embeddings of images. One of the methods includes obtaining training images; generating a plurality of triplets of training images; and training a neural network on each of the triplets to determine trained values of a plurality of parameters of the neural network, wherein training the neural network comprises, for each of the triplets: processing the anchor image in the triplet using the neural network to generate a numeric embedding of the anchor image; processing the positive image in the triplet using the neural network to generate a numeric embedding of the positive image; processing the negative image in the triplet using the neural network to generate a numeric embedding of the negative image; computing a triplet loss; and adjusting the current values of the parameters of the neural network using the triplet loss.

Generating numeric embeddings of images

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for object detection and representation in images. In one aspect, a method includes detecting occurrences of objects of a particular type in images captured within a first duration of time, and iteratively training an image embedding function to produce as output representations of features of the input images depicting occurrences of objects of the particular type, where similar representations of features are generated for images that depict the same instance of an object of a particular type captured within a specified duration of time, and dissimilar representations of features are generated for images that depict different instances of objects of the particular type.

Object detection and representation in images

Growing community of inventors

Santa Monica, CA, United States of America

Gerhard Florian Schroff