Nvidia Corporation

Jan Kautz

Sifei Liu

Ming-Yu Liu

Shalini De Mello

Kihwan Kim

Pavlo Molchanov

Xiaodong Yang

Varun Jampani

Jinwei Gu

Deqing Sun

Stephen Walter Tyree

Umar Iqbal

Ming-Hsuan Yang

Orazio Gallo

Alejandro Jose Troccoli

Xueting Li

David Patrick Luebke

Iuri Frosio

Zhiding Yu

Kari Pulli

Felix Heide

Nagilla Dikpal Reddy

Jonathan Tremblay

Dawid Stanislaw Pajak

Stan Birchfield

Arash Vahdat

Wonmin Byeon

Xitong Yang

Thomas Michael Breuel

Douglas Robert Lanman

Yi-Hsuan Tsai

Michael Stengel

Shalini Gupta

Eric Michael Whitmire

Ruben Eduardo Villegas

Kaan Aksit

Thang Hong To

Dikpal Reddy

Ben Boudaoud

Mushfiqur Rouf

Zhaoyang Lv

Guilin Liu

Patrick Wieschollek

Sergey Tulyakov

Wei-Chih Tu

Gregory P Meyer

Wei-Chih Hung

Artem Molchanov

Fanyi Xiao

Samarth Manoj Brahmbhatt

Benjamin David Eckart

Sina Honari

Timo Oskari Aila

Tero Tapani Karras

Sanja Fidler

Yijun Li

Anjul Patney

Stanley Thomas Birchfield

Charles T Loop

John R Zedlewski

Chao Liu

Jianyuan Min

Ting-Chun Wang

Karsten Julian Kreis

Arun Mohanray Mallya

Wenzheng Chen

Fu-Chung Huang

Anima Anandkumar

Rui Huang

Seung-Hwan Baek

Ye Yuan

Abhishek Haridas Badki

Rajeev Ranjan

Zian Wang

Hung-Yu Tseng

Huaijin Chen

Yun Rong Guo

Hang Su

Taihong Xiao

Hongxu Yin

Aysegul Dundar

Yang Zou

Cheng Xie

Jae Shin Yoon

Seonwook Park

Jiteng Mu

Donghoon Lee

Yang Fu

Jialiang Wang

Siva Karthik Mustikovela

Chen Liu

Tanmay Gupta

Yazhou Xing

Amrita Mazumdar

Mark Boss

Mayoore Selvarasa Jaiswal

Zhongheng Ren

Behrooz Mahasseni

Soumyadip Sengupta

A method, computer readable medium, and system are disclosed for action video generation. The method includes the steps of generating, by a recurrent neural network, a sequence of motion vectors from a first set of random variables and receiving, by a generator neural network, the sequence of motion vectors and a content vector sample. The sequence of motion vectors and the content vector sample are sampled by the generator neural network to produce a video clip.

System and method for content and motion controlled action video generation

Apparatuses, systems, and techniques are presented to determine visual properties of an object. In at least one embodiment, shape and visual properties of an object can be determined using differently illuminated images fed to a pipeline of neural networks.

Visual property determinations using one or more neural networks

One embodiment of a method includes applying a first generator model to a semantic representation of an image to generate an affine transformation, where the affine transformation represents a bounding box associated with at least one region within the image. The method further includes applying a second generator model to the affine transformation and the semantic representation to generate a shape of an object. The method further includes inserting the object into the image based on the bounding box and the shape.

Context-aware synthesis and placement of object instances

Apparatuses, systems, and techniques are presented to determine distance for one or more objects. In at least one embodiment, a disparity network is trained to determine distance data from input stereoscopic images using a loss function that includes at least one of a gradient loss term and an occlusion loss term.

Distance determinations using one or more neural networks

Apparatuses, systems, and techniques to detect object in images including digital representations of those objects. In at least one embodiment, one or more objects are detected in an image based, at least in part, on one or more pseudo-labels corresponding to said one or more objects.

Weakly-supervised object detection using one or more neural networks

One or more embodiments include receiving one or more frames of a live video captured by a video capturing device, wherein the one or more frames include a current frame that is most-recently captured, identifying a set of reference frames included in the one or more frames based on at least the current frame, wherein each frame in the set of reference frames has a different exposure level relative to the current frame, determining, using one or more neural networks, a set of missing details for one or more regions of the current frame based on the set of reference frames, generating an updated version of the current frame based on the set of details, and outputting the updated version of the current frame in real-time.

Hallucinating details for over-exposed pixels in videos using learned reference frame selection

Systems and methods to detect one or more segments of one or more objects within one or more images based, at least in part, on a neural network trained in an unsupervised manner to infer the one or more segments. Systems and methods to help train one or more neural networks to detect one or more segments of one or more objects within one or more images in an unsupervised manner.

Segmentation using an unsupervised neural network training technique

Systems and methods are disclosed related to synthetic bracketing for exposure correction. A deep learning based method and system produces a set of differently exposed images from a single input image. The images in the set may be combined to produce an output image with improved global and local exposure compared with the input image. An image encoder applies learned parameters to each input image to generate a set of image features including local exposure estimates for each of two or more regions of the input image and a low resolution latent representation of the input image. A decoder receives the local exposure estimates, the latent representation, and target enhancements that are processed to generate synthesized transformations. When applied to the input image, the synthesized transformations produce the set of transformed images. Each transformed image is a version of the input image synthesized to correspond to a respective target enhancement.

Synthetic bracketing for exposure correction

Systems and methods generate a hybrid lighting model for rendering objects within an image. The hybrid lighting model includes lighting effects attributed to a first source, such as the sun, and to a second source, such as spatially-varying effects of objects within the image. The hybrid lighting model may be generated for an input image and then one or more virtual objects may be rendered to appear as if part of the input image, where the hybrid lighting model is used to apply one or more lighting effects to the one or more virtual objects.

Differentiable object insertion using hybrid lighting volumes for synthetic data generation

Various embodiments enable a robot, or other autonomous or semi-autonomous device or system, to receive data involving the performance of a task in the physical world. The data can be provided as input to a perception network to infer a set of percepts about the task, which can correspond to relationships between objects observed during the performance. The percepts can be provided as input to a plan generation network, which can infer a set of actions as part of a plan. Each action can correspond to one of the observed relationships. The plan can be reviewed and any corrections made, either manually or through another demonstration of the task. Once the plan is verified as correct, the plan (and any related data) can be provided as input to an execution network that can infer instructions to cause the robot, and/or another robot, to perform the task.

Learning robotic tasks using one or more neural networks

A method and system are provided for tracking instances within a sequence of video frames. The method includes the steps of processing an image frame by a backbone network to generate a set of feature maps, processing the set of feature maps by one or more prediction heads, and analyzing the embedding features corresponding to a set of instances in two or more image frames of the sequence of video frames to establish a one-to-one correlation between instances in different image frames. The one or more prediction heads includes an embedding head configured to generate a set of embedding features corresponding to one or more instances of an object identified in the image frame. The method may also include training the one or more prediction heads using a set of annotated image frames and/or a plurality of sequences of unlabeled video frames.

Machine learning framework applied in a semi-supervised setting to perform instance tracking in a sequence of image frames

Apparatuses, systems, and techniques to identify orientations of objects within images. In at least one embodiment, one or more neural networks are trained to identify an orientations of one or more objects based, at least in part, on one or more characteristics of the object other than the object's orientation.

Training and inferencing using a neural network to predict orientations of objects in images

One embodiment of the present invention sets forth a technique for generating data. The technique includes sampling from a first distribution associated with the score-based generative model to generate a first set of values. The technique also includes performing one or more denoising operations via the score-based generative model to convert the first set of values into a first set of latent variable values associated with a latent space. The technique further includes converting the first set of latent variable values into a generative output.

Score-based generative modeling in latent space

Apparatuses, systems, and techniques to identify a shape or camera pose of a three-dimensional object from a two-dimensional image of the object. In at least one embodiment, objects are identified in an image using one or more neural networks that have been trained on objects of a similar category and a three-dimensional mesh template.

Self-supervised single-view 3D reconstruction via semantic consistency

In at least one embodiment, differentiable neural architecture search and reinforcement learning are combined under one framework to discover network architectures with desired properties such as high accuracy, low latency, or both. In at least one embodiment, an objective function for search based on generalization error prevents the selection of architectures prone to overfitting.

Machine-learning-based architecture search method for a neural network

Growing community of inventors

Lexington, MA, United States of America

Jan Kautz