Disney Enterprises, Inc.

Eidgenossische Technische Hochschule Zurich

Jeremy Riviere

Derek Edward Bradley

Paulo Fabiano Urnau Gotardo

Gaspard Zoss

Prashanth Chandran

Abhijeet Ghosh

Dominik Thabo Beeler

Yingyan Xu

Thabo D Beeler

Sebastian Winberg

Paulo Urnau Gotardo

Techniques are disclosed for generating photorealistic images of head portraits. A rendering application renders a set of images that include the skin of a face and corresponding masks indicating pixels associated with the skin in the images. An inpainting application performs a neural projection technique to optimize a set of parameters that, when input into a generator model, produces a set of projection images, each of which includes a head portrait in which (1) skin regions resemble the skin regions of the face in a corresponding rendered image; and (2) non-skin regions match the non-skin regions in the other projection images when the rendered set of images are standalone images, or transition smoothly between consecutive projection images in the case when the rendered set of images are frames of a video. The rendered images can then be blended with corresponding projection images to generate composite images that are photorealistic.

Techniques for enhancing skin renders using neural network projection for rendering completion

Various embodiments include a system for rendering an object, such as human skin or a human head, from captured appearance data. The system includes a processor executing a near field lighting reconstruction module. The system determines at least one of a three-dimensional (3D) position or a 3D orientation of a lighting unit based on a plurality of captured images of a mirror sphere. For each point light source in a plurality of point light sources included in the lighting unit, the system determines an intensity associated with the point light source. The system determines captures appearance data of the object, where the object is illuminated by the lighting unit. The system renders an image of the object based on the appearance data and the intensities associated with each point light source in the plurality of point light sources.

Techniques for improved lighting models for appearance capture

Various embodiments include a system for rendering an object, such as human skin or a human head, from captured appearance data comprising a plurality of texels. The system includes a processor executing a texture space indirect illumination module. The system determines texture coordinates of a vector originating from a first texel where the vector intersects a second texel. The system renders the second texel from the viewpoint of the first texel based on appearance data at the second texel. Based on the rendering of the second texel, the system determines an indirect lighting intensity incident to the first texel from the second texel. The system updates appearance data at the first texel based on a direct lighting intensity and the indirect lighting intensity. The system renders the first texel based on the updated appearance data at the first texel.

Techniques are disclosed for capturing facial appearance properties. In some examples, a facial capture system includes light source(s) that produce linearly polarized light, at least one camera that is cross-polarized with respect to the polarization of light produced by the light source(s), and at least one other camera that is not cross-polarized with respect to the polarization of the light produced by the light source(s). Images captured by the cross-polarized camera(s) are used to determine facial appearance properties other than specular intensity, such as diffuse albedo, while images captured by the camera(s) that are not cross-polarized are used to determine facial appearance properties including specular intensity. In addition, a coarse-to-fine optimization procedure is disclosed for determining appearance and detailed geometry maps based on images captured by the cross-polarized camera(s) and the camera(s) that are not cross-polarized.

Techniques for facial appearance capture

Techniques for object appearance capture

Embodiments of the present disclosure techniques for modeling and capturing the dynamic appearance of skin. These techniques can couple dynamic reflectance parameters for skin (albedo and specular reflectance) with dynamic geometry. The disclosed techniques allow for capture and modeling of the dynamic appearance of skin for an actor. The techniques can re-render the actor's face accurately to accurately model the appearance of skin including the albedo of skin that can change primarily due to blood flow. The techniques can also re-render the actor's face accurately under multiple different lighting conditions.

Growing community of inventors

Zurich, Switzerland

Jeremy Riviere