Nvidia Corporation

Microsoft Technology Licensing, LLC

Antonio Torralba Barriuso

Sanja Fidler

Seung Wook Kim

Huan Ling

Karsten Julian Kreis

Amlan Kar

Daiqing Li

Ming-Yu Liu

Wenzheng Chen

David Jesus Acuna Marrero

Aayush Prakash

Yuxuan Zhang

Jonah Philion

Ce Liu

Jun Gao

Robin Rombach

Yair Weiss

Bradley Brown

Approaches are presented for training an inverse graphics network. An image synthesis network can generate training data for an inverse graphics network. In turn, the inverse graphics network can teach the synthesis network about the physical three-dimensional (3D) controls. Such an approach can provide for accurate 3D reconstruction of objects from 2D images using the trained inverse graphics network, while requiring little annotation of the provided training data. Such an approach can extract and disentangle 3D knowledge learned by generative models by utilizing differentiable renderers, enabling a disentangled generative model to function as a controllable 3D 'neural renderer,' complementing traditional graphics renderers.

Neural rendering for inverse graphics generation

Apparatuses, systems, and techniques to use one or more neural networks to generate one or more images based, at least in part, on one or more spatially-independent features within the one or more images. In at least one embodiment, the one or more neural networks determine spatially-independent information and spatially-dependent information of the one or more images and process the spatially-independent information and the spatially-dependent information to generate the one or more spatially-independent features and one or more spatially-dependent features within the one or more images.

Generating frames for neural simulation using one or more neural networks

In various examples, a generative model is used to synthesize datasets for use in training a downstream machine learning model to perform an associated task. The synthesized datasets may be generated by sampling a scene graph from a scene grammar—such as a probabilistic grammar— and applying the scene graph to the generative model to compute updated scene graphs more representative of object attribute distributions of real-world datasets. The downstream machine learning model may be validated against a real-world validation dataset, and the performance of the model on the real-world validation dataset may be used as an additional factor in further training or fine-tuning the generative model for generating the synthesized datasets specific to the task of the downstream machine learning model.

Learning to generate synthetic datasets for training neural networks

In various examples, systems and methods are disclosed relating to generating a response from image and/or video input for image/video-based artificial intelligence (AI) systems and applications. Systems and methods are disclosed for a first model (e.g., a teacher model) distilling its knowledge to a second model (a student model). The second model receives a downstream image in a downstream task and generates at least one feature. The first model generates first features corresponding to an image which can be a real image or a synthetic image. The second model generates second features using the image as an input to the second model. Loss with respect to first features is determined. The second model is updated using the loss.

Unsupervised pre-training of neural networks using generative models

Apparatuses, systems, and techniques are presented to generate a simulated environment. In at least one embodiment, one or more neural networks are used to generate a simulated environment based, at least in part, on stored information associated with objects within the simulated environment.

Environment generation using one or more neural networks

Apparatuses, systems, and techniques are presented to generate digital content. In at least one embodiment, one or more neural networks are used to generate a three-dimensional voxel representation of a scene based, at least in part, upon a plurality of two-dimensional images of the scene.

Generating voxel representations using one or more neural networks

In various examples, high-precision semantic image editing for machine learning systems and applications are described. For example, a generative adversarial network (GAN) may be used to jointly model images and their semantic segmentations based on a same underlying latent code. Image editing may be achieved by using segmentation mask modifications (e.g., provided by a user, or otherwise) to optimize the latent code to be consistent with the updated segmentation, thus effectively changing the original, e.g., RGB image. To improve efficiency of the system, and to not require optimizations for each edit on each image, editing vectors may be learned in latent space that realize the edits, and that can be directly applied on other images with or without additional optimizations. As a result, a GAN in combination with the optimization approaches described herein may simultaneously allow for high precision editing in real-time with straightforward compositionality of multiple edits.

High-precision semantic image editing using neural networks for synthetic data generation systems and applications

In various examples, a generative model is used to synthesize datasets for use in training a downstream machine learning model to perform an associated task. The synthesized datasets may be generated by sampling a scene graph from a scene grammar&#x2014;such as a probabilistic grammar&#x2014;and applying the scene graph to the generative model to compute updated scene graphs more representative of object attribute distributions of real-world datasets. The downstream machine learning model may be validated against a real-world validation dataset, and the performance of the model on the real-world validation dataset may be used as an additional factor in further training or fine-tuning the generative model for generating the synthesized datasets specific to the task of the downstream machine learning model.

One or more aspects of the subject disclosure are directed towards identifying objects within an image via image searching/matching. In one aspect, an image is processed into bounding boxes, with the bounding boxes further processed to each surround a possible object. A sub-image of pixels corresponding to the bounding box is featurized for matching with tagged database images. The information (tags) associated with any matched images is processed to identify/categorize the sub-image and thus the object corresponding thereto.

Growing community of inventors

Somerville, MA, United States of America

Antonio Torralba Barriuso