Nvidia Corporation

Arash Vahdat

Karsten Julian Kreis

Jan Kautz

Sanja Fidler

Ming-Yu Liu

Xiaodong Yang

Arun Mohanray Mallya

Or Litany

Zan Gojcic

Tim Dockhorn

Francis Williams

Tianshi Cao

Tanmay Gupta

Xiaohui Zeng

Zhisheng Xiao

Sina Mohseni

Alex Bie

Jay Yadawa

Approaches presented herein provide for the unconditional generation of novel three dimensional (3D) object shape representations, such as point clouds or meshes. In at least one embodiment, a first denoising diffusion model (DDM) can be trained to synthesize a 1D shape latent from Gaussian noise, and a second DDM can be trained to generate a set of latent points conditioned on this 1D shape latent. The shape latent and set of latent points can be provided to a decoder to generate a 3D point cloud representative of a random object from among the object classes on which the models were trained. A surface reconstruction process may be used to generate a surface mesh from this generated point cloud. Such an approach can scale to complex and/or multimodal distributions, and can be highly flexible as it can be adapted to various tasks such as multimodal voxel- or text-guided synthesis.

Synthesizing three-dimensional shapes using latent diffusion models in content generation systems and applications

Apparatuses, systems, and techniques are presented to train and utilize one or more neural networks. A denoising diffusion generative adversarial network (denoising diffusion GAN) reduces a number of denoising steps during a reverse process. The denoising diffusion GAN does not assume a Gaussian distribution for large steps of the denoising process and applies a multi-model model to permit denoising with fewer steps. Systems and methods further minimize a divergence between a diffused real data distribution and a diffused generator distribution over several timesteps. Accordingly, various embodiments may enable faster sample generation, in which the samples are generated from noise using the denoising diffusion GAN.

Denoising diffusion generative adversarial networks

Systems and methods described relate to the synthesis of content using generative models. In at least one embodiment, a score-based generative model can use a stochastic differential equation with critically-damped Langevin diffusion to learn to synthesize content. During a forward diffusion process, noise can be introduced into a set of auxiliary (e.g., 'velocity') values for an input image to learn a score function. This score function can be used with the stochastic differential equation during a reverse diffusion denoising process to remove noise from the image to generate a reconstructed version of the input image. A score matching objective for the critically-damped Langevin diffusion process can require only the conditional distribution learned from the velocity data. A stochastic differential equation based integrator can then allow for efficient sampling from these critically-damped Langevin diffusion models.

Diffusion-based generative modeling for synthetic data generation systems and applications

One embodiment of the present invention sets forth a technique for generating data. The technique includes sampling from a first distribution associated with the score-based generative model to generate a first set of values. The technique also includes performing one or more denoising operations via the score-based generative model to convert the first set of values into a first set of latent variable values associated with a latent space. The technique further includes converting the first set of latent variable values into a generative output.

Score-based generative modeling in latent space

In at least one embodiment, differentiable neural architecture search and reinforcement learning are combined under one framework to discover network architectures with desired properties such as high accuracy, low latency, or both. In at least one embodiment, an objective function for search based on generalization error prevents the selection of architectures prone to overfitting.

Machine-learning-based architecture search method for a neural network

Apparatuses, systems, and techniques to identify out-of-distribution input data in one or more neural networks. In at least one embodiment, a technique includes training one or more neural networks to infer a plurality of characteristics about input information based, at least in part, on the one or more neural networks being independently trained to infer each of the plurality of characteristics about the input information.

Techniques for identification of out-of-distribution input data in neural networks

The disclosure provides a framework or system for learning visual representation using a large set of image/text pairs. The disclosure provides, for example, a method of visual representation learning, a joint representation learning system, and an artificial intelligence (AI) system that employs one or more of the trained models from the method or system. The AI system can be used, for example, in autonomous or semi-autonomous vehicles. In one example, the method of visual representation learning includes: (1) receiving a set of image embeddings from an image representation model and a set of text embeddings from a text representation model, and (2) training, employing mutual information, a critic function by learning relationships between the set of image embeddings and the set of text embeddings.

Joint representation learning from images and text

Apparatuses, systems, and techniques to train a generative model based at least in part on a private dataset. In at least one embodiment, the generative model is trained based at least in part on a differentially private Sinkhorn algorithm, for example, using backpropagation with gradient descent to determine a gradient of a set of parameters of the generative models and modifying the set of parameters based at least in part on the gradient.

Growing community of inventors

Mountain View, CA, United States of America

Arash Vahdat