University of Southern California

Disney Enterprises, Inc.

Imperial College Innovations Limited

Samsung Electronics Co., Ltd.

Abhijeet Ghosh

Paul E Debevec

Graham Fyffe

Derek Edward Bradley

Jeremy Riviere

Jay Busch

Do Kyoon Kim

Seon Min Rhee

Paulo Fabiano Urnau Gotardo

Hyun Jung Shim

Tae Hyun Rhee

Dominik Thabo Beeler

Xueming Yu

Pieter Peers

Mark T Bolas

Thabo D Beeler

Stefanos Zafeiriou

Valentin Deschaintre

Yiming Lin

Paulo Urnau Gotardo

Jen-Yuan Chiang

Matt Chiang

Paul Devebec

Yuliya Gitlina

Giuseppe Claudio Guarnera

Christos Kampouris

Daljit Singh Dhillon

Alexandros Lattas

Techniques are disclosed for capturing facial appearance properties. In some examples, a facial capture system includes light source(s) that produce linearly polarized light, at least one camera that is cross-polarized with respect to the polarization of light produced by the light source(s), and at least one other camera that is not cross-polarized with respect to the polarization of the light produced by the light source(s). Images captured by the cross-polarized camera(s) are used to determine facial appearance properties other than specular intensity, such as diffuse albedo, while images captured by the camera(s) that are not cross-polarized are used to determine facial appearance properties including specular intensity. In addition, a coarse-to-fine optimization procedure is disclosed for determining appearance and detailed geometry maps based on images captured by the cross-polarized camera(s) and the camera(s) that are not cross-polarized.

Techniques for facial appearance capture

A method of image processing includes receiving a first image of human skin. The first image corresponds to a first, uniform broadband illumination condition. The method also includes receiving a second image which has the same field of view and contents as the first image. The second image corresponds to a second illumination condition which comprises a uniform narrowband illumination condition. The method also includes processing the first and second images to fit parameter maps for a spectral bidirectional scattering surface reflectance distribution function skin model. The parameter maps include a modelled melanin concentration, a modelled haemoglobin concentration, a modelled melanin blend-type fraction and a modelled epidermal haemoglobin fraction. At least three of the parameter maps are independent.

Image capture and processing

A method of image processing is described. The method comprises receiving a set of at least three images of an object including at least two linearly-polarized images and at least one color image, wherein the three images have the same view of the object and are acquired under the same illumination condition in which either diffuse polarization or specular polarization dominates in surface reflectance, and wherein a set of Stokes parameters s, sand sis determinable from the at least three images. The method further comprises generating three-dimensional shape and spatially-varying reflectance of the object from the set of at least three images using a deep neural network trained with a plurality of sets of training images, each of the plurality of sets of training images including at least three training images including at least two linearly-polarized training images and at least one color image from which a respective set of Stokes parameters s, sand sis determinable and storing said three-dimensional shape and spatially-varying reflectance generated by the deep neural network.

Image processing for generating three-dimensional shape and spatially-varying reflectance of the object using a deep neural network

Techniques for object appearance capture

A method of image processing includes receiving a set of images of an object, the set of images including images corresponding to binary spherical gradient illumination along first, second and third mutually orthogonal axes and images corresponding to complementary binary spherical gradient illumination along the first, second and third axes. The method also includes determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of images, and/or determining a diffuse photometric normal map and a specular photometric normal map of the object based on the set of images.

Image processing for diffuse-specular separation

Embodiments of the present disclosure techniques for modeling and capturing the dynamic appearance of skin. These techniques can couple dynamic reflectance parameters for skin (albedo and specular reflectance) with dynamic geometry. The disclosed techniques allow for capture and modeling of the dynamic appearance of skin for an actor. The techniques can re-render the actor's face accurately to accurately model the appearance of skin including the albedo of skin that can change primarily due to blood flow. The techniques can also re-render the actor's face accurately under multiple different lighting conditions.

Techniques for capturing dynamic appearance of skin

An apparatus to measure surface orientation maps of an object may include a light source that is configured to illuminate the object with a controllable field of illumination. One or more cameras may be configured to capture at least one image of the object. A processor may be configured to process the image(s) to extract the reflectance properties of the object including an albedo, a reflection vector, a roughness, and/or anisotropy parameters of a specular reflectance lobe associated with the object. The controllable field of illumination may include limited-order Spherical Harmonics (SH) and Fourier Series (FS) illumination patterns with substantially similar polarization. The SH and FS illumination patterns are used with different light sources.

Specular object scanner for measuring reflectance properties of objects

Provided is a method and apparatus for realistically reproducing an eyeball that may verify and analyze a material property and a deformation property with respect to each of constituent portions of an eyeball and may render each of the constituent portions based on the analyzed priorities, thereby more realistically reproducing the eyeball.

Method and apparatus for realistically reproducing eyeball

A multiview face capture system may acquire detailed facial geometry with high resolution diffuse and specular photometric information from multiple viewpoints. A lighting system may illuminate a face with polarized light from multiple directions. The light may be polarized substantially parallel to a reference axis during a parallel polarization mode of operation and substantially perpendicular to the reference axis during a perpendicular polarization mode of operation. Multiple cameras may each capture an image of the face along a materially different optical axis and have a linear polarizer configured to polarize light traveling along its optical axis in a direction that is substantially parallel to the reference axis. A controller may cause each of the cameras to capture an image of the face while the lighting system is in the parallel polarization mode of operation and again while the lighting system is in the perpendicular polarization mode of operation.

Multiview face capture using polarized spherical gradient illumination

A controllable lighting system may include a plurality of light source groups, a group controller for each light source group, a master controller, and a network communication system. Each group controller may be configured to control the light sources in its light source group based on a group control command. The master controller may be configured to receive a master control command relating to the light sources and to issue a group control command to each of the group controllers that collectively effectuate compliance with the master control command. The network communication system may be configured to communicate the group control commands from the master controller to the group controllers.

Illumination sphere with intelligent LED lighting units in scalable daisy chain with interchangeable filters

A method and apparatus for realistically expressing teeth performs rendering on a shape model of the teeth using a hybrid normal map based on materials and characteristics of the teeth in order to more realistically express the teeth.

Apparatus and method for realistically expressing teeth

A system for estimating the specular roughness of points on a surface of an object may include a lighting system, an image capture system and a computer processing system. The lighting system may be configured to illuminate the surface of the object at different times with different illumination patterns. Each illumination pattern may illuminate the surface from a plurality of different directions and form an intensity gradient having an order of no more than two. The image capture system may be configured to capture an image of the surface of the object when illuminated by each of the different illumination patterns at each of the different times. The computer processing system may be configured to compute the specular roughness of each point on the surface of the object based on the images captured by the image capture system.

Estimating spectral distribution of reflections from object surface based on low frequency illumination

Techniques are described for modeling layered facial reflectance consisting of specular reflectance, single scattering, and shallow and deep subsurface scattering. Parameters of appropriate reflectance models can be estimated for each of these layers, e.g., from just 20 photographs recorded in a few seconds from a single view-point. Spatially-varying specular reflectance and single-scattering parameters can be extracted from polarization-difference images under spherical and point source illumination. Direct-indirect separation can be employed to decompose the remaining multiple scattering observed under cross-polarization into shallow and deep scattering components to model the light transport through multiple layers of skin. Appropriate diffusion models can be matched to the extracted shallow and deep scattering components for different regions on the face. The techniques were validated by comparing renderings of subjects to reference photographs recorded from novel viewpoints and under novel illumination conditions. Related geometry acquisition systems and software products are also described.

Growing community of inventors

London, United Kingdom

Abhijeet Ghosh