Nvidia Corporation

Nvidia Corporaton

Ting-Chun Wang

Ming-Yu Liu

Arun Mohanray Mallya

Samuli Matias Laine

David Patrick Luebke

Timo Oskari Aila

Tero Tapani Karras

Jaakko Tapani Lehtinen

Miika Samuli Aittala

Karan Sapra

Andrew J Tao

Bryan Catanzaro

Guilin Liu

Fitsum A Reda

Jan Kautz

John R Zedlewski

Shiqiu Liu

Zhiding Yu

Kevin Shih

Aysegul Dundar

Jose Rafael Valle Da Costa

Tero Karras

Tim Brooks

Jaakko Lehtinen

Brandon Rowlett

Siddharth Gururani

Apparatuses, systems, and techniques are presented to synthesize consistent images or video. In at least one embodiment, one or more neural networks are used to generate one or more second images based, at least in part, on one or more point cloud representations of one or more first images.

Video synthesis using one or more neural networks

A neural network architecture is disclosed for performing image in-painting using partial convolution operations. The neural network processes an image and a corresponding mask that identifies holes in the image utilizing partial convolution operations, where the mask is used by the partial convolution operation to zero out coefficients of the convolution kernel corresponding to invalid pixel data for the holes. The mask is updated after each partial convolution operation is performed in an encoder section of the neural network. In one embodiment, the neural network is implemented using an encoder-decoder framework with skip links to forward representations of the features at different sections of the encoder to corresponding sections of the decoder.

Image in-painting for irregular holes using partial convolutions

Apparatuses, systems, and techniques are presented to generate digital content. In at least one embodiment, one or more neural networks are used to generate video information based at least in part upon voice information and a combination of image features and facial landmarks corresponding to one or more images of a person.

Speech-driven animation using one or more neural networks

Apparatuses, systems, and techniques for texture synthesis from small input textures in images using convolutional neural networks. In at least one embodiment, one or more convolutional layers are used in conjunction with one or more transposed convolution operations to generate a large textured output image from a small input textured image while preserving global features and texture, according to various novel techniques described herein.

Techniques to use a neural network to expand an image

Apparatuses, systems, and techniques are presented to reconstruct one or more images. In at least one embodiment, one or more neural networks are used to generate one or more images of one or more objects based, at least in part, on input indicating motion of the one or more objects.

Generating images of object motion using one or more neural networks

A latent code defined in an input space is processed by the mapping neural network to produce an intermediate latent code defined in an intermediate latent space. The intermediate latent code may be used as appearance vector that is processed by the synthesis neural network to generate an image. The appearance vector is a compressed encoding of data, such as video frames including a person's face, audio, and other data. Captured images may be converted into appearance vectors at a local device and transmitted to a remote device using much less bandwidth compared with transmitting the captured images. A synthesis neural network at the remote device reconstructs the images for display.

Generative adversarial neural network assisted video reconstruction

Generative adversarial neural network assisted compression and broadcast

Generative adversarial neural network assisted reconstruction

Generative adversarial neural network assisted video compression and broadcast

A style transfer neural network may be used to generate stylized synthetic images, where real images provide the style (e.g., seasons, weather, lighting) for transfer to synthetic images. The stylized synthetic images may then be used to train a recognition neural network. In turn, the trained neural network may be used to predict semantic labels for the real images, providing recognition data for the real images. Finally, the real training dataset (real images and predicted recognition data) and the synthetic training dataset are used by the style transfer neural network to generate stylized synthetic images. The training of the neural network, prediction of recognition data for the real images, and stylizing of the synthetic images may be repeated for a number of iterations. The stylization operation more closely aligns a covariate of the synthetic images to the covariate of the real images, improving accuracy of the recognition neural network.

Domain stylization using a neural network model

Growing community of inventors

Santa Clara, CA, United States of America

Ting-Chun Wang