Pinscreen, Inc.

Koki Nagano

Hao Li

Lingyu Wei

Liwen Hu

Jaewoo Seo

Jens Fursund

Zejian Wang

Huiwen Luo

Shunsuke Saito

A system, method, and apparatus for generating a normalized three-dimensional model of a human face from a single unconstrained two-dimensional image of the human face. The system includes a processor that executes instructions including receiving the single unconstrained two-dimensional image of the human face, using an inference network to determine an inferred normalized three-dimensional model of the human face based on the single unconstrained two-dimensional image of the human face, and using a refinement network to iteratively determine the normalized three-dimensional model of the human face with a neutral expression and unshaded albedo textures under diffuse lighting conditions based on the inferred normalized three-dimensional model of the human face.

Normalized three-dimensional avatar synthesis and perceptual refinement

A system, method, and apparatus for generating a normalization of a single two-dimensional image of an unconstrained human face. The system receives the single two-dimensional image of the unconstrained human face, generates an undistorted face based on the unconstrained human face by removing perspective distortion from the unconstrained human face via a perspective undistortion network, generates an evenly lit face based on the undistorted face by normalizing lighting of the undistorted face via a lighting translation network, and generates a frontalized and neutralized expression face based on the evenly lit face via an expression neutralization network.

Normalization of facial images using deep neural networks

A system and method for generating real-time facial animation is disclosed. The system relies upon pre-generating a series of key expression images from a single neutral image using a pre-trained generative adversarial neural network. The key expression images are used to generate a set of FACS expressions and associated textures which may be applied to a three-dimensional model to generate facial animation. The FACS expressions and textures may be provided to a mobile device to enable that mobile device to generate convincing three-dimensional avatars in real-time with convincing animation in a processor non-intensive way through a blending process using the pre-determined FACS expressions and textures.

Real-time avatars using dynamic textures

A system for generating three-dimensional facial models including photorealistic hair and facial textures includes creating a facial model with reliance upon neural networks based upon a single two-dimensional input image. The photorealistic hair is created by finding a subset of similar three-dimensional polystrip hairstyles from a large database of polystrip hairstyles, selecting the most-alike polystrip hairstyle, deforming that polystrip hairstyle to better fit the hair of the two-dimensional image. Then, collisions and bald spots are corrected, and suitable textures are applied. Finally, the facial model and polystrip hairstyle are combined into a final three-dimensional avatar.

Growing community of inventors

Los Angeles, CA, United States of America

Koki Nagano