Nvidia Corporation

Other

University of Toronto

Amlan Kar

Sanja Fidler

David Jesus Acuna Marrero

Huan Ling

Jun Gao

Karan Singh

Wenzheng Chen

Antonio Torralba Barriuso

Maria Shugrina

Ming-Yu Liu

Aayush Prakash

Seung Wook Kim

Daiqing Li

David Acuna Marrero

Jeevan Devaranjan

In various examples, systems and methods are disclosed relating to generating a response from image and/or video input for image/video-based artificial intelligence (AI) systems and applications. Systems and methods are disclosed for a first model (e.g., a teacher model) distilling its knowledge to a second model (a student model). The second model receives a downstream image in a downstream task and generates at least one feature. The first model generates first features corresponding to an image which can be a real image or a synthetic image. The second model generates second features using the image as an input to the second model. Loss with respect to first features is determined. The second model is updated using the loss.

Unsupervised pre-training of neural networks using generative models

A rule set or scene grammar can be used to generate a scene graph that represents the structure and visual parameters of objects in a scene. A renderer can take this scene graph as input and, with a library of content for assets identified in the scene graph, can generate a synthetic image of a scene that has the desired scene structure without the need for manual placement of any of the objects in the scene. Images or environments synthesized in this way can be used to, for example, generate training data for real world navigational applications, as well as to generate virtual worlds for games or virtual reality experiences.

Unsupervised learning of scene structure for synthetic data generation

In various examples, a generative model is used to synthesize datasets for use in training a downstream machine learning model to perform an associated task. The synthesized datasets may be generated by sampling a scene graph from a scene grammar&#x2014;such as a probabilistic grammar&#x2014;and applying the scene graph to the generative model to compute updated scene graphs more representative of object attribute distributions of real-world datasets. The downstream machine learning model may be validated against a real-world validation dataset, and the performance of the model on the real-world validation dataset may be used as an additional factor in further training or fine-tuning the generative model for generating the synthesized datasets specific to the task of the downstream machine learning model.

Learning to generate synthetic datasets for training neural networks

The present invention relates generally to object annotation, specifically to polygonal annotations of objects. Described are methods of annotating an object including steps of receiving an image depicting an object, generating a set of image features using a CNN encoder implemented on one or more computers, and producing a polygon object annotation via a recurrent decoder or a Graph Neural Network. The recurrent decoder may include a recurrent neural network, a graph neural network or a gated graph neural network. A system for annotating an object and a method of training an object annotation system are also described.

Systems and methods for polygon object annotation and a method of training an object annotation system

There is provided a system and method for color representation generation. In an aspect, the method includes: receiving three base colors; receiving a patchwork parameter; and generating a color representation having each of the three base colors at a vertex of a triangular face, the triangular face having a color distribution therein, the color distribution discretized into discrete portions, the amount of discretization based on the patchwork parameter, each discrete portion having an interpolated color determined to be a combination of the base colors at respective coordinates of such discrete portion. In further aspects, one or more color representations are generated based on an input image and can be used to modify colors of a reconstructed image.

Method and system for color representation generation

Systems and methods for polygon object annotation and a method of training and object annotation system

Growing community of inventors

Toronto, Canada

Amlan Kar