Nvidia Corporation

Aayush Prakash

David Jesus Acuna Marrero

Stanley Thomas Birchfield

Jonathan Tremblay

Varun Jampani

Thang Hong To

Mark A Brophy

Cem Anil

Sanja Fidler

Amlan Kar

Gavriel State

Marc Teva Law

Shoubhik Debnath

Jean-Francois Lafleche

Eric Cameracci

Ming-Yu Liu

Antonio Torralba Barriuso

In various examples, a generative model is used to synthesize datasets for use in training a downstream machine learning model to perform an associated task. The synthesized datasets may be generated by sampling a scene graph from a scene grammar—such as a probabilistic grammar— and applying the scene graph to the generative model to compute updated scene graphs more representative of object attribute distributions of real-world datasets. The downstream machine learning model may be validated against a real-world validation dataset, and the performance of the model on the real-world validation dataset may be used as an additional factor in further training or fine-tuning the generative model for generating the synthesized datasets specific to the task of the downstream machine learning model.

Learning to generate synthetic datasets for training neural networks

Approaches are presented for training and using scene graph generators for transfer learning. A scene graph generation technique can decompose a domain gap into individual types of discrepancies, such as may relate to appearance, label, and prediction discrepancies. These discrepancies can be reduced, at least in part, by aligning the corresponding latent and output distributions using one or more gradient reversal layers (GRLs). Label discrepancies can be addressed using self-pseudo-statistics collected from target data. Pseudo statistic-based self-learning and adversarial techniques can be used to manage these discrepancies without the need for costly supervision from a real-world dataset.

Scene graph generation for unlabeled data

Training deep neural networks requires a large amount of labeled training data. Conventionally, labeled training data is generated by gathering real images that are manually labelled which is very time-consuming. Instead of manually labelling a training dataset, domain randomization technique is used generate training data that is automatically labeled. The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks. In an embodiment, the generated training data includes synthetic input images generated by rendering three-dimensional (3D) objects of interest in a 3D scene. In an embodiment, the generated training data includes synthetic input images generated by rendering 3D objects of interest on a 2D background image. The 3D objects of interest are objects that a neural network is trained to detect and/or label.

Neural network model trained using generated synthetic images

In various examples, a generative model is used to synthesize datasets for use in training a downstream machine learning model to perform an associated task. The synthesized datasets may be generated by sampling a scene graph from a scene grammar&#x2014;such as a probabilistic grammar&#x2014;and applying the scene graph to the generative model to compute updated scene graphs more representative of object attribute distributions of real-world datasets. The downstream machine learning model may be validated against a real-world validation dataset, and the performance of the model on the real-world validation dataset may be used as an additional factor in further training or fine-tuning the generative model for generating the synthesized datasets specific to the task of the downstream machine learning model.

Generation of synthetic images for training a neural network model

Growing community of inventors

Toronto, Canada

Aayush Prakash