Microsoft Technology Licensing, LLC

Zhejiang University

Faceunity Technology Co., Ltd.

International Business Machines Corporation

Microsoft Corporation

Kun Zhou

Baining Guo

Heung-Yeung Shum

John Michael Snyder

Menglei Chai

Xin Huang

Minmin Gong

Zhong Ren

Lvdi Wang

Weiwei Xu

Yanlin Weng

Hou Qiming

Li-Yi Wei

Stephen S Lin

Stephen Ssu-te Lin

Xinguo Liu

Xiaohan Shi

Qiming Hou

Xi Wang

Ashraf A Michail

Gregory David Swedberg

Adrian Secchia

Changxi Zheng

Jingdan Zhang

Hongzhi Wu

Yu Chen

Rui Wang

Peter-Pike Johannes Sloan

Lifeng Wang

Yasuyuki Matsushita

Rui Li

Jiaping Wang

Yanyun Chen

Jin Huang

Yaohua Hu

Li Jia Chen

Jun Wang

Heung Yeung Shum

Michiel Van De Panne

Jianwei Han

Chen Cao

Kangkang Yin

Yizhou Yu

Qiuju Geng

Bo Sun

Guo Baining

Yizhong Zhang

Shum Heung-Yeung

Chunji Yin

Jiahao Geng

Kaizhang Kang

Yujin Chai

Xiaohe Ma

A freestyle acquisition method for a high-dimensional material, belonging to the field of computer graphics and computer vision. The learning of material information is transformed into a geometric learning problem on an unstructured point cloud, and a plurality of acquisition results in different lighting and view directions form a high-dimensional point cloud, each point in the point cloud being a vector formed by an image measurement value and pose information of an object during image capture. According to the method, information of unstructured views can be effectively aggregated from the high-dimensional point cloud which is disordered, irregular, uneven in distribution and limited in precision, and the material attribute with high quality is reconstructed.

Freestyle acquisition method for high-dimensional material

A method for obtaining a normal vector, a geometry and a material of a three-dimensional object based on a neural network is provided. The present disclosure provides, based on an idea of 'actively irradiating an object with a number of specific patterns, capturing photos at the same time, and obtaining a normal vector of an object by calculating the obtained photos', an acquisition method combined with a neural network. Further, the method uses the obtained normal vector to optimize a model of the object. This method can also obtain material feature information while obtaining the normal vector. Finally, a high-quality geometric result and a high-quality material acquisition result are obtained jointly. The number of illumination patterns obtained by using this method is small, and the normal vector obtained by the method has high accuracy and the method is not limited to a specific acquisition device.

Methods for obtaining normal vector, geometry and material of three-dimensional objects based on neural network

A method for generating a facial animation from a single image is provided. The method is mainly divided into four steps: generation of facial feature points in an image, global two-dimensional deformation of the image, optimization of details of a facial area, and generation of texture of an oral cavity area. The present disclosure can generate a facial animation in real time according to a change of the facial feature points, and the animation quality reaches a level of current state-of-art facial image animation technology. The present disclosure can be used in a series of applications, such as facial image editing, portrait animation generation based on a single image, and facial expression editing in videos.

Method for generating facial animation from single image

The present disclosure discloses a speech-driven facial animation generation method. The method is mainly divided into six steps: extracting speech features, collecting frequency information, summarizing time information, decoding action features, driving a facial model, and sliding a signal window. The present disclosure can drive, according to an input speech audio signal, any facial model in real time under a particular delay to generate animation. The quality of the animation reaches the currently most advanced speech animation technology level, and has the characteristics of light weight and good robustness. The present disclosure can be used to generate speech animation under different scenes, such as VR virtual social networking, and virtual speech assistants and games.

Speech-driven facial animation generation method

Provided is a single-image-based fully automatic three-dimensional (3D) hair modeling method. The method mainly includes four steps: generation of hair image training data, hair segmentation and growth direction estimation based on a hierarchical depth neural network, generation and organization of 3D hair exemplars, and data-driven 3D hair modeling. The method can automatically and robustly generate a complete high quality 3D model of which the quality reaches the level of the currently most advanced user interaction-based technology. The method can be used in a series of applications, such as hair style editing in portrait images, browsing of hair style spaces, and searching for Internet images of similar hair styles.

Method for single-image-based fully automatic three-dimensional hair modeling

A method for efficiently visualizing parallel vector data rapidly renders high-quality complex vector graphics. The present invention provides a scanline algorithm for calculating and rendering on contour lines, which parallel-rasterizing on the contour lines. The algorithm firstly rasterizes the contour lines into segments corresponding to output image pixels. On each segment, color values are analytically calculated, or approximately obtained by a sampling algorithm. Contour rasterizing is efficiently completed in parallel. By analytical calculation or 32-bit sampling, high quality results are obtained. The present invention adapts parallel prefixes and algorithms in a scanline direction to obtain covering information for each pixel and generate filled areas. Finally, the contour line segments and the filled areas of the vector graphics are rendered in the output image. The present invention is entirely based on parallel many-core computing devices for real-time high-quality vector graphic rendering with hardware acceleration.

Method for efficient parallel visualizing vector graphic

Disclosed is a real-time motion simulation method for hair and object collisions, which is based on a small amount of pre-computation training data and generates a self-adaptive simplified model for virtual hair style for real-time selection and interpolation and collision correction, thereby realizing real-time high-quality motion simulation for hair-object collisions. The method comprises the following steps: 1) reduced model pre-computation: based on pre-computation simulation data, selecting representative hairs and generating a reduced model; 2) real-time animation and interpolation: clustering the representative hairs simulated in real time; selecting the reduced model and interpolating; and 3) collision correction: detecting collision and applying a correction force on the representative hairs to correct the collisions. The present invention proposed a real-time simulation method for hair-object collision, which achieves similar effect as off-line simulation and reduces the computation time cost.

Real-time animation method for hair-object collisions

A user interface for a touchscreen display device including a technique for cursor positioning, with respect to: (i) a touchscreen display upon which the cursor is displayed; and (ii) displayable content (also sometimes herein referred to as a 'display area') which is also, at least in part, displayed on the touchscreen display. Based on a desired position to which the activated display area is moved while a cursor remains static on the display screen, a placement position at which the cursor is to be placed with respect to the displayable content is determined.

Cursor positioning on display screen

A calculable 3D color printing method is used for generating, on a 3D object surface, a user-specified color texture pattern, including: performing modeling and simulated calculation with respect to a motion and a deformation of a PVA film in a traditional water transfer printing process to obtain a mapping function between each of pixels on the PVA film and points on an object surface; and calculating, by using the mapping function, a texture pattern to print on the PVA film. This disclosure further includes a set of mechanical devices and a 3D vision system for realizing precise and controllable water transfer printing which precisely prints a user-specified texture pattern onto an object surface. This disclosure further includes a multiple water transfer printing method, including: partitioning a complex object surface into a plurality of regions, and separately coloring each region, until eventually completing coloring of the entire object surface.

Calculable three-dimensional color printing method

The invention discloses a method for image and video virtual hairstyle modeling, including: performing data acquisition for a target subject by using a digital device and obtaining a hairstyle region from an image by segmenting; obtaining a uniformly distributed static hairstyle model which conforms to the original hairstyle region by solving an orientation ambiguity problem of an image hairstyle orientation field, calculating a movement of the hairstyle in a video by tracing a movement of a head model and estimating non-rigid deformation, generating a dynamic hairstyle model in every moment during the moving process, so that the dynamic hairstyle model fits the real movement of the hairstyle in the video naturally. The method is used to perform virtual 3D model reconstruction with physical rationality for individual hairstyles in single-views and video sequences, and widely applied in creating virtual characters and many hairstyle editing applications for images and videos.

Method for image and video virtual hairstyle modeling

The invention discloses a method for single-view hair modeling and portrait editing. The method is capable of 3D structure reconstruction for individual's hairstyle in an input image, and it requires only a small amount of user inputs to bring about a variety of portrait editing functions; after steps of image preprocessing, 3D head model reconstruction, 2D strands extraction and 3D hairstyle reconstruction, the method finally achieves portrait editing functions such as portrait pop-ups, hairstyle replacements, hairstyle editing, etc.; the invention discloses a method for creating a 3D hair model from a single portrait view for the first time, thereby bringing about a series of practical portrait hairstyle editing functions, of which the effect is superior to methods in the prior art, and having features such as simple interactions and highly efficient calculations.

Method for single-view hair modeling and portrait editing

The invention discloses a method for real-time face animation based on single video camera. The method tracks 3D locations of face feature points in real time by adopting a single video camera, and parameterizes head poses and facial expressions according to the 3D locations, finally may map these parameters into an avatar to drive face animation of an animation character. The present invention may achieve a real time speed by merely adopting a usual video camera of the user instead of an advanced acquisition equipment; the present invention may process all kinds of wide-angle rotations, translation and exaggerated expressions of faces accurately; the present invention may also work under different illumination and background environments, which include indoor and sunny outdoor.

Method for real-time face animation based on single video camera

Described is a technology by which a user interacts with a surface representative of a point cloud data to correct for imperfect scan data. The surface is reconstructed based on the interaction. Real time viewing of the image is facilitated by parallel surface reconstruction. For example, the user may draw strokes to reduce topological ambiguities in poorly-sampled areas. An algorithm automatically adds new oriented sample points to the original point cloud based on the user interaction. Then a new isosurface is generated for the augmented point cloud. The user also may specify the geometry of missing areas of the surface. The user copies a set of points from another point cloud, and places the points around the target area. A new isosurface is then generated.

User-guided surface reconstruction

Described is a technology by which a ray tracer incorporates a GPU-based kd-tree builder for rendering arbitrary dynamic scenes. For each frame, the ray tracer builds a kd-tree for the scene geometry. The ray tracer spawns and traces eye rays, reflective and refractive rays, and shadow rays. For each ray to be traced, the ray tracer walks through the kd-tree until it reaches leaf nodes and associated triangles. When a ray passes through both sides of a splitting plane, the 'far' sub-tree is pushed into the stack and the 'near' sub-tree is traversed first.

Ray tracing on graphics hardware using kd-trees

Described is a technology by which a GPU-based photon mapping mechanism/algorithm uses a kd-tree to render arbitrary dynamic scenes. For each frame, the mechanism emits and traces a set of photons into the scene. When a photon hits a surface, it can either be reflected, transmitted, or absorbed based on the surface material. Once photon tracing is done, a kd-tree is built for the stored photons. To estimate the radiance value at an arbitrary surface point, the k-nearest photons are located and filtered. The photon tracing and photon kd-tree construction, as well as the radiance estimation using k-nearest neighbor (KNN) searches are performed on graphics hardware, e.g., a GPU. In one example, only caustic photons are traced, whereby a photon is terminated and stored once it hits a diffuse surface.

Photon mapping on graphics hardware using kd-trees

Growing community of inventors

Beijing, China

Kun Zhou