Nvidia Corporation

Other

David Jesus Acuna Marrero

Sanja Fidler

Varun Jampani

Aayush Prakash

Stanley Thomas Birchfield

Jonathan Tremblay

Thang Hong To

Amlan Kar

Mark A Brophy

Cem Anil

Jun Gao

Wenzheng Chen

Huan Ling

Towaki Alan Takikawa

Ming-Yu Liu

Antonio Torralba Barriuso

Various types of image analysis benefit from a multi-stream architecture that allows the analysis to consider shape data. A shape stream can process image data in parallel with a primary stream, where data from layers of a network in the primary stream is provided as input to a network of the shape stream. The shape data can be fused with the primary analysis data to produce more accurate output, such as to produce accurate boundary information when the shape data is used with semantic segmentation data produced by the primary stream. A gate structure can be used to connect the intermediate layers of the primary and shape streams, using higher level activations to gate lower level activations in the shape stream. Such a gate structure can help focus the shape stream on the relevant information and reduces any additional weight of the shape stream.

Shape fusion for image analysis

Training deep neural networks requires a large amount of labeled training data. Conventionally, labeled training data is generated by gathering real images that are manually labelled which is very time-consuming. Instead of manually labelling a training dataset, domain randomization technique is used generate training data that is automatically labeled. The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks. In an embodiment, the generated training data includes synthetic input images generated by rendering three-dimensional (3D) objects of interest in a 3D scene. In an embodiment, the generated training data includes synthetic input images generated by rendering 3D objects of interest on a 2D background image. The 3D objects of interest are objects that a neural network is trained to detect and/or label.

Neural network model trained using generated synthetic images

In various examples, a generative model is used to synthesize datasets for use in training a downstream machine learning model to perform an associated task. The synthesized datasets may be generated by sampling a scene graph from a scene grammar&#x2014;such as a probabilistic grammar&#x2014;and applying the scene graph to the generative model to compute updated scene graphs more representative of object attribute distributions of real-world datasets. The downstream machine learning model may be validated against a real-world validation dataset, and the performance of the model on the real-world validation dataset may be used as an additional factor in further training or fine-tuning the generative model for generating the synthesized datasets specific to the task of the downstream machine learning model.

Learning to generate synthetic datasets for training neural networks

The present invention relates generally to object annotation, specifically to polygonal annotations of objects. Described are methods of annotating an object including steps of receiving an image depicting an object, generating a set of image features using a CNN encoder implemented on one or more computers, and producing a polygon object annotation via a recurrent decoder or a Graph Neural Network. The recurrent decoder may include a recurrent neural network, a graph neural network or a gated graph neural network. A system for annotating an object and a method of training an object annotation system are also described.

Systems and methods for polygon object annotation and a method of training an object annotation system

Generation of synthetic images for training a neural network model

Systems and methods for polygon object annotation and a method of training and object annotation system

Growing community of inventors

Toronto, Canada

David Jesus Acuna Marrero