Lucasfilm Entertainment Company Ltd.

Lucas Digital Ltd.

Cary Phillips

Frederic P Pighin

Steve Sullivan

Michael Koperwas

Kiran Bhat

Christian Rouet

Eduardo Hueso

Christopher David Twigg

Rachel Rose

Jason Smith

Jung-Seung Hong

John R Anderson

Leandro F Estebecorena

Ronald Fedkiw

Pablo Helman

Jeffery Bruce Yost

Per Karefelt

Stéphane Grabli

James Robert Tooley

Nicolas Popravka

Michael Bao

Adam Ferrall-Nunge

A method of transferring a facial expression from a subject to a computer generated character that includes receiving a plate with an image of the subject's facial expression, a three-dimensional parameterized deformable model of the subject's face where different facial expressions of the subject can be obtained by varying values of the model parameters, a model of a camera rig used to capture the plate, and a virtual lighting model that estimates lighting conditions when the image on the plate was captured. The method can solve for the facial expression in the plate by executing a deformation solver to solve for at least some parameters of the deformable model with a differentiable renderer and shape from shading techniques, using, as inputs, the three-dimensional parameterized deformable model, the model of the camera rig and the virtual lighting model over a series of iterations to infer geometry of the facial expression and generate a final facial mesh using the set of parameter values of the deformable model which result in a facial expression that closely matches the expression of the subject in the plate.

On-set facial performance capture and transfer to a three-dimensional computer-generated model

A system includes a computing device that includes a memory configured to store instructions. The system also includes a processor configured to execute the instructions to perform a method that includes receiving multiple representations of one or more expressions of an object. Each of the representations includes position information attained from one or more images of the object. The method also includes producing an animation model from one or more groups of controls that respectively define each of the one or more expressions of the object as provided by the multiple representations. Each control of each group of controls has an adjustable value that defines the geometry of at least one shape of a portion of the respective expression of the object. Producing the animation model includes producing one or more corrective shapes if the animation model is incapable of accurately presenting the one or more expressions of the object as provided by the multiple representations.

Facial animation models

A system includes a computing device that includes a memory and a processor configured to execute instructions to perform a method that includes receiving multiple representations of one or more expressions of an object. Each representation includes position information attained from one or more images of the object. The method also includes producing an animation model from one or more groups of controls that respectively define each of the one or more expressions of the object as provided by the multiple representations. Each control of each group of controls has an adjustable value that defines the geometry of at least one shape of a portion of the respective expression of the object. Producing the animation model includes producing one or more corrective shapes if the animation model is incapable of accurately presenting the one or more expressions of the object as provided by the multiple representations.

The subject matter of this specification can be embodied in, among other things, a method that includes determining a transform of a portion of constituent components of a source shape. The transform includes one or more states for the portion of constituent components of the source shape. The method also includes accessing a mapping function that associates the one or more states with one or more controls for a target shape, where the one or more controls configured for access by a user for manipulating constituent components of the target shape. The method includes outputting a transform for the target shape based on the one or more controls associated with the transformed constituent components of the source shape.

Animation control retargeting

A system includes a computer system capable of representing one or more animated characters. The computer system includes a blendshape manager that combines multiple blendshapes to produce the animated character. The computer system also includes an expression manager to respectively adjust one or more control parameters associated with each of the plurality of blendshapes for adjusting an expression of the animated character. The computer system also includes a corrective element manager that applies one or more corrective elements to the combined blendshapes based upon at least one of the control parameters. The one or more applied corrective elements are adjustable based upon one or more of the control parameters absent the introduction of one or more additional control parameters.

Controlling animated character expression

A computer-implemented method includes identifying a representation of a feature of an animated character by inverting a skinned representation of the feature in one position. The inversion includes a non-linear inversion of the skinned representation of the feature. The method also includes skinning the identified feature representation to produce the animated character in another position.

Inversion of post-skinning features

Controlling animated character expressions

Systems and methods are described, which create a mapping from a space of a source object (e.g., source facial expressions) to a space of a target object (e.g., target facial expressions). In certain implementations, the mapping is learned based a training set composed of corresponding shapes (e.g. facial expressions) in each space. The user can create the training set by selecting expressions from, for example, captured source performance data, and by sculpting corresponding target expressions. Additional target shapes (e.g., target facial expressions) can be interpolated and extrapolated from the shapes in the training set to generate corresponding shapes for potential source shapes (e.g., facial expressions).

Animation retargeting

Among other disclosed subject matter, a computer-implemented method for generating a surface representation of an item includes identifying, for a point on an item in an animation process, at least first and second transformation points corresponding to respective first and second transformations of the point. Each of the first and second transformations represents an influence on a location of the point of respective first and second joints associated with the item. The method includes determining an axis for a cylindrical coordinate system using the first and second transformations. The method includes performing an interpolation of the first and second transformation points in the cylindrical coordinate system to obtain an interpolated point. The method includes recording the interpolated point in a surface representation of the item in the animation process.

Generating a surface representation of an item

A computer-implemented method includes identifying a representation of a feature of an animated character by inverting an skinned representation of the feature in one position. The inversion includes a non-linear inversion of the skinned representation of the feature. The method also includes skinning the identified feature representation to produce the animated character in another position.

The subject matter of this specification can be embodied in, among other things, a system that includes a correspondence generator that determines a distance-based mapping between a set of points in a source object and a point in a target object and stores the distance-based mapping in a surface map. The system also includes an attribute transfer module that transfers surface attributes that determine an appearance or deformation of the set of points in the source object to the point in the target object using the surface map. The system also includes an attribute editor that receives a selection of a region of the target object and applies an interpolation algorithm that determines new values for surface attributes in the selected region from neighboring points in the target object.

Growing community of inventors

Moss Beach, CA, United States of America

Cary Phillips