Unity Technologies Sf

Weta Digital Limited

Byung Kuk Choi

Gergely Klár

Karan Singh

Stephen Cullingford

Albrecht Steinmetz

Haekwang Eom

An image generation system defines a set of deformation handles having an associated set of one or more control parameters, obtains a set of object points representative of a virtual object, maps the set of object points to a plurality of key pose states, wherein the key pose state is represented by a key pose state data structure corresponding to a key pose that specifies control parameters to place the set of the deformation handles to coincide with the set of object points, determines corresponding key pose state data structures, receives a set of deformations to be applied to the set of deformation handles, interpolates poses among an interpolated key pose state set to form a current pose state based on the set of deformations, and adjusts the virtual object based on the interpolated key pose state set.

Object deformation with bindings and deformers interpolated from key poses

A computer-implemented method and interface provides visualization for a volume and/or a change in a volume of a virtual object, the representation usable in a user interface, comprising a first representation of the virtual object represented by a surface positioned in a three-dimensional (3D) virtual space, a bounding object input that defines a bounding object relative to the 3D virtual space, a plurality of local volumes of the virtual object, wherein a local volume of the plurality of local volumes is computed for geometry based on a bounding object feature associated with the bounding object and based on at least some vertices of the corresponding face, wherein at least some of the plurality of local volumes are aggregated to determine a global volume for the set of polygonal faces, and providing for display of a first visual indicator representing a value of the global volume.

Computer graphics interface using visual indicator representing object global volume and/or global volume changes and method therefore

An animation system wherein a machine learning model is adopted to learn a transformation relationship between facial muscle movements and skin surface movements. For example, for the skin surface representing 'smile,' the transformation model derives movement vectors relating to what facial muscles are activated, what are the muscle strains, what is the joint movement, and/or the like. Such derived movement vectors may be used to simulate the skin surface “smile.”.

Systems and methods for building a muscle-to-skin transformation in computer animation

An animation system wherein scanned facial expressions are processed to form muscle models based on live actors combines muscle models over a plurality of live actors to form a facial rig usable for generating expressions based on specification of a strain vector and a control vector of a muscle model for varying characters corresponding to live actors.

Generating a facial rig for use in animating a computer-generated character based on facial scans and muscle models of multiple live actors

A character rig may be representable as a data structure specifying a plurality of articulated character parts, an element tree specifying relations between character parts, and a set of constraints on the character parts. After receiving rotoscoping movement input data corresponding to attempted alignments of movements of at least some of the character parts with elements moving in a captured live action scene, a rotoscoping constraints may be received. The rotoscoping constraint may include at least a first constraint on the character rig other than a second constraint specified by the data structure of the character rig, Thereafter, rig movement inputs for a second set of character parts distinct from the first set of character parts may be accepted and the character rig may be moved according to the rig movement inputs while constrained by the rotoscoping constraints.

Method for operating a character rig in an image-generation system using constraints on reference nodes

An animation system wherein a machine learning model is adopted to generate animated facial actions based on parameters obtained from a live actor. Specifically, the anatomical structure such as a facial muscle topology and a skull surface that are specific to the live actor may be used. A skull surface that is specific to a live actor based on facial scans of the live actor and generic tissue depth data. For example, the facial scans of the live actor may provide a skin surface topology of the live actor, based on which the skull surface underneath the skin surface can be derived by 'offsetting' the skin surface with corresponding soft tissue depth at different sampled points on the skin surface.

Systems and methods for generating a skull surface for computer animation

An animation system wherein a machine learning model is adopted to generate animated facial actions based on parameters obtained from a live actor. Specifically, the anatomical structure such as a facial muscle topology and a skull surface topology that are specific to the live actor may be used. A skull surface that is specific to a live actor based on facial scans of the live actor and generic tissue depth data. For example, the facial scans of the live actor may provide a skin surface topology of the live actor, based on which the skull surface underneath the skin surface can be derived by 'offsetting' the skin surface with corresponding soft tissue depth at different sampled points on the skin surface.

An animation system wherein a machine learning model is adopted to generate animated facial actions based on parameters obtained from a live actor. Specifically, the anatomical structure such as a facial muscle topology and a skull surface that are specific to the live actor may be used. A muscle structure of simplified 'pseudo' muscles that approximate the actual muscle topology but with reduced degree of freedom is determined to improve computational efficiency.

Systems and methods for building a pseudo-muscle topology of a live actor in computer animation

An animation system wherein scanned facial expressions are processed to form muscle models that can be used to generate expressions based on specification of a strain vector and a control vector of the muscle model.

Generating an animation rig for use in animating a computer-generated character based on facial scans of an actor and a muscle model

In an image processing system, a scan of an actor is converted to a high-resolution two-dimensional map, which is converted to low-resolution map and to a facial rig model. Manipulations of the facial rig create a modified facial rig. A new low-resolution two-dimensional map can be obtained of the modified facial rig and a neural network can be used to generate a new high-resolution two-dimensional map that can be used to generate a mesh that is a mesh of the scan, modified by the manipulations of the facial rig.

Facial model mapping with a neural network trained on varying levels of detail of facial scans

An animation system wherein a machine learning model is adopted to learn a transformation relationship between facial muscle movements and skin surface movements. For example, for the skin surface representing 'smile,' the transformation model derives movement vectors relating to what facial muscles are activated, what are the muscle strains, what is the joint movement, and/or the like. Such derived movement vectors may be used to simulate the skin surface 'smile.'

Growing community of inventors

Wellington, New Zealand

Byung Kuk Choi