Microsoft Technology Licensing, LLC

Google Inc.

Matthew Alun Brown

Gang Hua

Sofien Bouaziz

Daniel Goldman

Simon Winder

Rohit Kumar Pandey

Grant Evan Schindler

Ricardo Martin Brualla

Simon A J Winder

Seyed Mohammad Mehdi Sajjadi

Raviteja Vemulapalli

Richard Hartley

Jonathan Chung-Kuan Huang

Tal Remez

Example aspects of the present disclosure are directed to systems and methods that enable weakly-supervised learning of instance segmentation by applying a cut-and-paste technique to training of a generator model included in a generative adversarial network. In particular, the present disclosure provides a weakly-supervised approach to object instance segmentation. In some implementations, starting with known or predicted object bounding boxes, a generator model can learn to generate object masks by playing a game of cut-and-paste in an adversarial learning setup.

Learning to segment via cut-and-paste

Systems and methods are described for generating a plurality of three-dimensional (3D) proxy geometries of an object, generating, based on the plurality of 3D proxy geometries, a plurality of neural textures of the object, the neural textures defining a plurality of different shapes and appearances representing the object, providing the plurality of neural textures to a neural renderer, receiving, from the neural renderer and based on the plurality of neural textures, a color image and an alpha mask representing an opacity of at least a portion of the object, and generating a composite image based on the pose, the color image, and the alpha mask.

Generative latent textured proxies for object category modeling

The present disclosure provides systems and methods to increase resolution of imagery. In one example embodiment, a computer-implemented method includes obtaining a current low-resolution image frame. The method includes obtaining a previous estimated high-resolution image frame, the previous estimated high-resolution frame being a high-resolution estimate of a previous low-resolution image frame. The method includes warping the previous estimated high-resolution image frame based on the current low-resolution image frame. The method includes inputting the warped previous estimated high-resolution image frame and the current low-resolution image frame into a machine-learned frame estimation model. The method includes receiving a current estimated high-resolution image frame as an output of the machine-learned frame estimation model, the current estimated high-resolution image frame being a high-resolution estimate of the current low-resolution image frame.

Frame-recurrent video super-resolution

To render the comparison of image patches more efficient, the data of an image patch can be projected into a smaller-dimensioned subspace, resulting in a descriptor of the image patch. The projection into the descriptor subspace is known as a linear discriminant embedding, and can be performed with reference to a linear discriminant embedding matrix. The linear discriminant embedding matrix can be constructed from projection vectors that maximize those elements that are shared by matching image patches or that are used to distinguish non-matching image patches, while also minimizing those elements that are common to non-matching image patches or that distinguish matching image patches. The determination of such projection vectors can be limited such that only orthogonal vectors comprise the linear discriminant embedding matrix. The determination of the linear discriminant embedding matrix can likewise be constrained to avoid overfitting to training data.

Local image descriptors using linear discriminant embedding

A local image descriptor generation technique that produces a descriptor for an image patch is presented. The technique generally involves smoothing the pixels of the image patch, followed by employing a transformation to produce a transform vector for each of a set of sample points spaced across the image patch. The transform vectors are weighted and spatially accumulated to produce a prescribed number of linearly summed vectors. The linearly summed vectors are concatenated to form a raw local image descriptor, which is normalized to produce a finalized descriptor for the image patch.

Local image descriptors

A location recognition technique that involves using a query image to identify a depicted location is presented. In addition to the query image, there is also a pre-constructed database of features which are associated with images of known locations. The technique matches features derived from the query image to the database features using a specialized vocabulary tree, which is referred to as an informative feature vocabulary tree. The informative feature vocabulary tree is specialized because it was generated using just those database features that have been deemed informative of known locations. The aforementioned matching features are used to identify a known location image that matches the query image. The location associated with that known location image is then deemed to be the location depicted in the query image.

Location recognition using informative feature vocabulary trees

A focal length estimation technique is presented that estimates the focal length used to capture each of a pair of overlapping images taken by a camera rotating about its center. When it is known that the focal length is the same for both images, it is estimated using just two pairs of corresponding points between the images. When it is either unknown whether the focal lengths are the same or it is known that they vary, the focal length of each image is estimated using just three pairs of corresponding points between the images.

Growing community of inventors

Seattle, WA, United States of America

Matthew Alun Brown