Lucasfilm Entertainment Company Ltd.

Pablo Helman

Leandro F Estebecorena

Per Karefelt

John M Levin

Stephane Grabli

John Knoll

Cary Phillips

Ronald Fedkiw

Jeffery Bruce Yost

Stéphane Grabli

Adam Ferrall-Nunge

Michael Bao

Jeffery Yost

Leandro Estebecorena

Embodiments of the disclosure provide systems and methods for motion capture to generate content (e.g., motion pictures, television programming, videos, etc.). An actor or other performing being can have multiple markers on his or her face that are essentially invisible to the human eye, but that can be clearly captured by camera systems of the present disclosure. Embodiments can capture the performance using two different camera systems, each of which can observe the same performance but capture different images of that performance. For instance, a first camera system can capture the performance within a first light wavelength spectrum (e.g., visible light spectrum), and a second camera system can simultaneously capture the performance in a second light wavelength spectrum different from the first spectrum (e.g., invisible light spectrum such as the IR light spectrum). The images captured by the first and second camera systems can be combined to generate content.

Camera systems for motion capture

A method of transferring a facial expression from a subject to a computer generated character that includes receiving a plate with an image of the subject's facial expression, a three-dimensional parameterized deformable model of the subject's face where different facial expressions of the subject can be obtained by varying values of the model parameters, a model of a camera rig used to capture the plate, and a virtual lighting model that estimates lighting conditions when the image on the plate was captured. The method can solve for the facial expression in the plate by executing a deformation solver to solve for at least some parameters of the deformable model with a differentiable renderer and shape from shading techniques, using, as inputs, the three-dimensional parameterized deformable model, the model of the camera rig and the virtual lighting model over a series of iterations to infer geometry of the facial expression and generate a final facial mesh using the set of parameter values of the deformable model which result in a facial expression that closely matches the expression of the subject in the plate.

On-set facial performance capture and transfer to a three-dimensional computer-generated model

Systems and methods for motion capture

Growing community of inventors

San Rafael, CA, United States of America

Pablo Helman