Pinscreen, Inc.

University of Southern California

Shunsuke Saito

Hao Li

Liwen Hu

Jens Fursund

Lingyu Wei

Koki Nagano

Jaewoo Seo

Tianye Li

Cosimo Wei

Devices and methods for single-view 3D hair modeling are disclosed. The method for single-view 3D hair modeling includes training, by a neural network processor, a volumetric autoencoder to encode a plurality of 3D hairstyles into latent features, and to generate an output based on the latent features. The method for single-view 3D hair modeling includes training, by the neural network processor, an embedding network to determine hair coefficients of a single hairstyle from an input image. The method for single-view 3D hair modeling includes receiving, by the neural network processor, the input image. The method for single-view 3D hair modeling includes synthesizing, by the neural network processor, hair strands to generate a single-view 3D model of the single hairstyle based on the volumetric autoencoder, the embedding network, and the input image.

3D hair synthesis using volumetric variational autoencoders

A system for generating three-dimensional facial models including photorealistic hair and facial textures includes creating a facial model with reliance upon neural networks based upon a single two-dimensional input image. The photorealistic hair is created by finding a subset of similar three-dimensional polystrip hairstyles from a large database of polystrip hairstyles, selecting the most-alike polystrip hairstyle, deforming that polystrip hairstyle to better fit the hair of the two-dimensional image. Then, collisions and bald spots are corrected, and suitable textures are applied. Finally, the facial model and polystrip hairstyle are combined into a final three-dimensional avatar.

Avatar digitization from a single image for real-time rendering

A method for generating three-dimensional facial models and photorealistic textures from inferences using deep neural networks relies upon generating a low frequency and a high frequency albedo map of the full and partial face, respectively. Then, the high frequency albedo map may be used for comparison with correlation matrices generated by a neural network trained by a large scale, high-resolution facial dataset with simulated partial visibility. The corresponding correlation matrices of the complete facial textures can then be retrieved. Finally, a full facial texture map may be synthesized, using convex combinations of the correlation matrices. A photorealistic facial texture for the three-dimensional face rendering can be obtained through optimization using the deep neural network and a loss function that incorporates the blended target correlation matrices.

Photorealistic facial texture inference using deep neural networks

There is disclosed a system and method of performing facial recognition from RGB image data. The method includes generating a lower-resolution image from the RGB image data, performing a convolution of the lower-resolution image data to derive a probability map identifying probable facial regions and a probable non-facial regions, and performing a first deconvolution on the lower-resolution image using a bilinear interpolation layer to derive a set of coarse facial segments. The method further includes performing a second deconvolution on the lower-resolution image using a series of unpooling, deconvolution, and rectification layers to derive a set of fine facial segments, concatenating the set of coarse facial segments to the set of fine facial segments to create an image matrix made up of a set of facial segments, and generating a binary facial mask identifying probable facial regions and probable non-facial regions from the image matrix.

Growing community of inventors

Los Angeles, CA, United States of America

Shunsuke Saito