Nvidia Corporation

Jan Kautz

Sifei Liu

Shalini De Mello

Ming-Yu Liu

Kihwan Kim

Pavlo Molchanov

Xiaodong Yang

Varun Jampani

Jinwei Gu

Deqing Sun

Stephen Walter Tyree

Umar Iqbal

Ming-Hsuan Yang

Orazio Gallo

Alejandro Jose Troccoli

Xueting Li

David Patrick Luebke

Iuri Frosio

Zhiding Yu

Kari Pulli

Felix Heide

Nagilla Dikpal Reddy

Jonathan Tremblay

Dawid Stanislaw Pajak

Arash Vahdat

Stan Birchfield

Wonmin Byeon

Xitong Yang

Thomas Michael Breuel

Douglas Robert Lanman

Yi-Hsuan Tsai

Michael Stengel

Shalini Gupta

Eric Michael Whitmire

Ruben Eduardo Villegas

Kaan Aksit

Thang Hong To

Dikpal Reddy

Ben Boudaoud

Mushfiqur Rouf

Zhaoyang Lv

Guilin Liu

Patrick Wieschollek

Wei-Chih Tu

Gregory P Meyer

Wei-Chih Hung

Artem Molchanov

Fanyi Xiao

Samarth Manoj Brahmbhatt

Benjamin David Eckart

Sina Honari

Timo Oskari Aila

Tero Tapani Karras

Sanja Fidler

Yijun Li

Anjul Patney

Stanley Thomas Birchfield

Charles T Loop

John R Zedlewski

Chao Liu

Jianyuan Min

Karsten Julian Kreis

Ting-Chun Wang

Arun Mohanray Mallya

Wenzheng Chen

Fu-Chung Huang

Anima Anandkumar

Rui Huang

Seung-Hwan Baek

Ye Yuan

Abhishek Haridas Badki

Rajeev Ranjan

Zian Wang

Hung-Yu Tseng

Huaijin Chen

Yun Rong Guo

Hang Su

Taihong Xiao

Hongxu Yin

Aysegul Dundar

Yang Zou

Sergey Tulyakov

Cheng Xie

Jae Shin Yoon

Seonwook Park

Jiteng Mu

Donghoon Lee

Yang Fu

Jialiang Wang

Siva Karthik Mustikovela

Chen Liu

Tanmay Gupta

Yazhou Xing

Amrita Mazumdar

Mayoore Selvarasa Jaiswal

Zhongheng Ren

Behrooz Mahasseni

Soumyadip Sengupta

One embodiment of a method includes applying a first generator model to a semantic representation of an image to generate an affine transformation, where the affine transformation represents a bounding box associated with at least one region within the image. The method further includes applying a second generator model to the affine transformation and the semantic representation to generate a shape of an object. The method further includes inserting the object into the image based on the bounding box and the shape.

Context-aware synthesis and placement of object instances

Apparatuses, systems, and techniques are presented to determine distance for one or more objects. In at least one embodiment, a disparity network is trained to determine distance data from input stereoscopic images using a loss function that includes at least one of a gradient loss term and an occlusion loss term.

Distance determinations using one or more neural networks

Apparatuses, systems, and techniques to detect object in images including digital representations of those objects. In at least one embodiment, one or more objects are detected in an image based, at least in part, on one or more pseudo-labels corresponding to said one or more objects.

Weakly-supervised object detection using one or more neural networks

One or more embodiments include receiving one or more frames of a live video captured by a video capturing device, wherein the one or more frames include a current frame that is most-recently captured, identifying a set of reference frames included in the one or more frames based on at least the current frame, wherein each frame in the set of reference frames has a different exposure level relative to the current frame, determining, using one or more neural networks, a set of missing details for one or more regions of the current frame based on the set of reference frames, generating an updated version of the current frame based on the set of details, and outputting the updated version of the current frame in real-time.

Hallucinating details for over-exposed pixels in videos using learned reference frame selection

Systems and methods to detect one or more segments of one or more objects within one or more images based, at least in part, on a neural network trained in an unsupervised manner to infer the one or more segments. Systems and methods to help train one or more neural networks to detect one or more segments of one or more objects within one or more images in an unsupervised manner.

Segmentation using an unsupervised neural network training technique

Systems and methods are disclosed related to synthetic bracketing for exposure correction. A deep learning based method and system produces a set of differently exposed images from a single input image. The images in the set may be combined to produce an output image with improved global and local exposure compared with the input image. An image encoder applies learned parameters to each input image to generate a set of image features including local exposure estimates for each of two or more regions of the input image and a low resolution latent representation of the input image. A decoder receives the local exposure estimates, the latent representation, and target enhancements that are processed to generate synthesized transformations. When applied to the input image, the synthesized transformations produce the set of transformed images. Each transformed image is a version of the input image synthesized to correspond to a respective target enhancement.

Synthetic bracketing for exposure correction

Systems and methods generate a hybrid lighting model for rendering objects within an image. The hybrid lighting model includes lighting effects attributed to a first source, such as the sun, and to a second source, such as spatially-varying effects of objects within the image. The hybrid lighting model may be generated for an input image and then one or more virtual objects may be rendered to appear as if part of the input image, where the hybrid lighting model is used to apply one or more lighting effects to the one or more virtual objects.

Differentiable object insertion using hybrid lighting volumes for synthetic data generation

Various embodiments enable a robot, or other autonomous or semi-autonomous device or system, to receive data involving the performance of a task in the physical world. The data can be provided as input to a perception network to infer a set of percepts about the task, which can correspond to relationships between objects observed during the performance. The percepts can be provided as input to a plan generation network, which can infer a set of actions as part of a plan. Each action can correspond to one of the observed relationships. The plan can be reviewed and any corrections made, either manually or through another demonstration of the task. Once the plan is verified as correct, the plan (and any related data) can be provided as input to an execution network that can infer instructions to cause the robot, and/or another robot, to perform the task.

Learning robotic tasks using one or more neural networks

A method and system are provided for tracking instances within a sequence of video frames. The method includes the steps of processing an image frame by a backbone network to generate a set of feature maps, processing the set of feature maps by one or more prediction heads, and analyzing the embedding features corresponding to a set of instances in two or more image frames of the sequence of video frames to establish a one-to-one correlation between instances in different image frames. The one or more prediction heads includes an embedding head configured to generate a set of embedding features corresponding to one or more instances of an object identified in the image frame. The method may also include training the one or more prediction heads using a set of annotated image frames and/or a plurality of sequences of unlabeled video frames.

Machine learning framework applied in a semi-supervised setting to perform instance tracking in a sequence of image frames

Apparatuses, systems, and techniques to identify orientations of objects within images. In at least one embodiment, one or more neural networks are trained to identify an orientations of one or more objects based, at least in part, on one or more characteristics of the object other than the object's orientation.

Training and inferencing using a neural network to predict orientations of objects in images

One embodiment of the present invention sets forth a technique for generating data. The technique includes sampling from a first distribution associated with the score-based generative model to generate a first set of values. The technique also includes performing one or more denoising operations via the score-based generative model to convert the first set of values into a first set of latent variable values associated with a latent space. The technique further includes converting the first set of latent variable values into a generative output.

Score-based generative modeling in latent space

Apparatuses, systems, and techniques to identify a shape or camera pose of a three-dimensional object from a two-dimensional image of the object. In at least one embodiment, objects are identified in an image using one or more neural networks that have been trained on objects of a similar category and a three-dimensional mesh template.

Self-supervised single-view 3D reconstruction via semantic consistency

In at least one embodiment, differentiable neural architecture search and reinforcement learning are combined under one framework to discover network architectures with desired properties such as high accuracy, low latency, or both. In at least one embodiment, an objective function for search based on generalization error prevents the selection of architectures prone to overfitting.

Machine-learning-based architecture search method for a neural network

Embodiments of the present disclosure relate to learning dense correspondences for images. Systems and methods are disclosed that disentangle structure and texture (or style) representations of GAN synthesized images by learning a dense pixel-level correspondence map for each image during image synthesis. A canonical coordinate frame is defined and a structure latent code for each generated image is warped to align with the canonical coordinate frame. In sum, the structure associated with the latent code is mapped into a shared coordinate space (canonical coordinate space), thereby establishing correspondences in the shared coordinate space. A correspondence generation system receives the warped coordinate correspondences as an encoded image structure. The encoded image structure and a texture latent code are used to synthesize an image. The shared coordinate space enables propagation of semantic labels from reference images to synthesized images.

Learning dense correspondences for images

In order to determine accurate three-dimensional (3D) models for objects within a video, the objects are first identified and tracked within the video, and a pose and shape are estimated for these tracked objects. A translation and global orientation are removed from the tracked objects to determine local motion for the objects, and motion infilling is performed to fill in any missing portions for the object within the video. A global trajectory is then determined for the objects within the video, and the infilled motion and global trajectory are then used to determine infilled global motion for the object within the video. This enables the accurate depiction of each object as a 3D pose sequence for that model that accounts for occlusions and global factors within the video.

Performing occlusion-aware global 3D pose and shape estimation of articulated objects

Growing community of inventors

Lexington, MA, United States of America

Jan Kautz