Microsoft Technology Licensing, LLC

Hsiang-Tao Wu

Xin Tong

Yang Liu

Fan Yang

Zicheng Liu

Fang Wen

David G Molyneaux

Jay P Kapur

Emily Yang

Baoyuan Wang

Eric Chang

Noranart Vesdapunt

Yangang Wang

Wenbin Zhu

Zeyu Chen

Techniques performed by a data processing system for reconstructing a three-dimensional (3D) model of the face of a human subject herein include obtaining source data comprising a two-dimensional (2D) image, three-dimensional (3D) image, or depth information representing a face of a human subject. Reconstructing the 3D model of the face also includes generating a 3D model of the face of the human subject based on the source data by analyzing the source data to produce a coarse 3D model of the face of the human subject, and refining the coarse 3D model through free form deformation to produce a fitted 3D model. The coarse 3D model may be a 3D Morphable Model (3DMM), and the coarse 3D model may be refined through free-form deformation in which the deformation of the mesh is limited by applying an as-rigid-as-possible (ARAP) deformation constraint.

Reinforced differentiable attribute for 3D face reconstruction

The present disclosure provides method, apparatus and system for 3-dimension (3D) face tracking. The method for 3D face tracking may comprise: obtaining a 2-dimension (2D) face image; performing a local feature regression on the 2D face image to determine 3D face representation parameters corresponding to the 2D face image; and generating a 3D facial mesh and corresponding 2D facial landmarks based on the determined 3D face representation parameters. The present disclosure may improve tracking accuracy and reduce memory cost, and accordingly may be effectively applied in broader application scenarios.

Method, apparatus and system for 3D face tracking

A method for constructing an avatar of a human subject includes acquiring a depth map of the subject, obtaining a virtual skeleton of the subject based on the depth map, and harvesting from the virtual skeleton a set of characteristic metrics. Such metrics correspond to distances between predetermined points of the virtual skeleton. In this example method, the characteristic metrics are provided as input to an algorithm trained using machine learning. The algorithm may be trained using a human model in a range of poses, and a range of human models in a single pose, to output a virtual body mesh as a function of the characteristic metrics. The method also includes constructing a virtual head mesh distinct from the virtual body mesh, with facial features resembling those of the subject, and connecting the virtual body mesh to the virtual head mesh.

Avatar construction using depth camera

Growing community of inventors

Beijing, China

Hsiang-Tao Wu