Disney Enterprises, Inc.

Eth Zurich (eidgenossische Technische Hochschule Zurich)

Wisconsin Alumni Research Foundation

Eftychios Sifakis

Derek Edward Bradley

Gaspard Zoss

Dominik Thabo Beeler

Andrew Selle

Joseph M Teran

Aleka McAdams

Paulo Fabiano Urnau Gotardo

Prashanth Chandran

Rasmus Tamstorf

Nathan M Mitchell

Michael S Doescher

Kelly Ward

Techniques are disclosed for re-aging images of faces and three-dimensional (3D) geometry representing faces. In some embodiments, an image of a face, an input age, and a target age, are input into a re-aging model, which outputs a re-aging delta image that can be combined with the input image to generate a re-aged image of the face. In some embodiments, 3D geometry representing a face is re-aged using local 3D re-aging models that each include a blendshape model for finding a linear combination of sample patches from geometries of different facial identities and generating a new shape for the patch at a target age based on the linear combination. In some embodiments, 3D geometry representing a face is re-aged by performing a shape-from-shading technique using re-aged images of the face captured from different viewpoints, which can optionally be constrained to linear combinations of sample patches from local blendshape models.

Techniques for re-aging faces in images and video frames

A modeling engine generates a prediction model that quantifies and predicts secondary dynamics associated with the face of a performer enacting a performance. The modeling engine generates a set of geometric representations that represents the face of the performer enacting different facial expressions under a range of loading conditions. For a given facial expression and specific loading condition, the modeling engine trains a Machine Learning model to predict how soft tissue regions of the face of the performer change in response to external forces applied to the performer during the performance. The modeling engine combines different expression models associated with different facial expressions to generate a prediction model. The prediction model can be used to predict and remove secondary dynamics from a given geometric representation of a performance or to generate and add secondary dynamics to a given geometric representation of a performance.

Data-driven extraction and composition of secondary dynamics in facial performance capture

A removal model is trained to predict secondary dynamics associated with an individual enacting a performance. For a given sequence of frames that includes an individual enacting a performance and secondary dynamics, a retargeting application identifies a set of rigid points that correspond to skeletal regions of the individual and a set of non-rigid points that correspond to non-skeletal region of the individual. For each frame in the sequence of frames, the application applies the removal model that takes as inputs a velocity history of a non-rigid point and a velocity history of the rigid points in a temporal window around the frame, and outputs a delta vector for the non-rigid point indicating a displacement for reducing secondary dynamics in the frame. In addition, a trained synthesis model can be applied to determine a delta vector for every non-rigid point indicating displacements for adding new secondary dynamics.

Techniques for removing and synthesizing secondary dynamics in facial performance capture

In accordance with some embodiments, systems, methods and media for simulating deformation of an elastic body are provided. In some embodiments, a method comprises: determining for each macroblock, a stiffness matrix Kof a portion of a model of a non-linear elastic solid partitioned into cells; converting Kinto block form to include a submatrix Kfor nodes between internal cells of a first macroblock; determining at least a portion of K; receiving input corresponding to force applied to cells of the model; determining displacements of exterior nodes of the first macroblock using the input and the portion of K; determining displacements of interior nodes of the first macroblock using the input and the displacements of exterior nodes; determining updated positions of the cells based on the displacements of the exterior nodes; and, causing the model to be presented using the updated positions.

Systems, methods and, media for simulating deformations of nonlinear elastic bodies

A novel algorithmic framework is presented for the simulation of hyperelastic soft tissues that drastically improves each aspect discussed above compared to existing techniques. The approach is robust to large deformation (even inverted configurations) and extremely stable by virtue of careful treatment of linearization. Additionally, a new multigrid approach is presented to efficiently support hundreds of thousands of degrees of freedom (rather than the few thousands typical of existing techniques) in a production environment. Furthermore, these performance and robustness improvements are guaranteed in the presence of both collision and quasistatic/implicit time stepping techniques. The result is a significant advance in the applicability of hyperelastic simulation to skeleton driven character skinning.

Efficient elasticity for character skinning

A computer graphic system and methods for simulating hair is provided. In accordance with aspects of the disclosure a method for hybrid hair simulation using a computer graphics system is provided. The method includes generating a plurality of modeled hair strands using a processor of the computer graphics system. Each hair strand includes a plurality of particles and a plurality of spring members coupled in between the plurality of particles. The method also includes determining a first position and a first velocity for each particle in the plurality of modeled hair strands using the processor and coarsely modeling movement of the plurality of modeled hair strands with a continuum fluid solver. Self-collisions of the plurality of modeled hair strands are computed with a discrete collision model using the processor.

Growing community of inventors

Verona, WI, United States of America

Eftychios Sifakis