Adobe, Inc.

Mixamo, Inc.

Facebook, Inc.

Roblox Corporation

Emiliano Gambaretto

Stefano Koenig Corazza

Kalyan Krishna Sunkavalli

Jonathan Eisenmann

Sunil Sharadchandra Hadap

Mehmet Yumer

Yannick Hold-Geoffroy

Xiaohui Shen

Vladimir Kim

Marc-André Gardner

Edilson De Aguiar

Radomir Mech

Matthew David Fisher

Duygu Ceylan Aksit

Kevin Wampler

Noam Aigerman

Siddhartha Chaudhuri

Giorgio Gori

Prasanna Vasudevan

Kiran Bhat

Qingnan Zhou

Daniel Babcock

Jinsong Zhang

Charles Piña

Vincent Petrella

Simone Guggiari

David Brown

Implementations described herein relate to methods, systems, and computer-readable media to display a virtual character within a virtual environment on a display device. In some implementations, a method can include obtaining an input pose of the virtual character, wherein the virtual character is based on a rig that comprises a plurality of joints, receiving an indication of one or more of a position and an orientation of a target end effector located on the rig, determining an output pose for the rig wherein the determining comprises calculating a respective orientation and position of one or more joints of the plurality of joints of the rig based on the position of the target end effector and rotation constraints of a plurality of joints of a reference rig, and displaying the virtual character in the output pose on the display device.

Determination and display of inverse kinematic poses of virtual characters in a virtual environment

Embodiments of the present invention are directed towards intuitive editing of three-dimensional models. In embodiments, salient geometric features associated with a three-dimensional model defining an object are identified. Thereafter, feature attributes associated with the salient geometric features are identified. A feature set including a plurality of salient geometric features related to one another is generated based on the determined feature attributes (e.g., properties, relationships, distances). An editing handle can then be generated and displayed for the feature set enabling each of the salient geometric features within the feature set to be edited in accordance with a manipulation of the editing handle. The editing handle can be displayed in association with one of the salient geometric features of the feature set.

Intuitive editing of three-dimensional models

Systems and techniques for estimating illumination from a single image are provided. An example system may include a neural network. The neural network may include an encoder that is configured to encode an input image into an intermediate representation. The neural network may also include an intensity decoder that is configured to decode the intermediate representation into an output light intensity map. An example intensity decoder is generated by a multi-phase training process that includes a first phase to train a light mask decoder using a set of low dynamic range images and a second phase to adjust parameters of the light mask decoder using a set of high dynamic range image to generate the intensity decoder.

High dynamic range illumination estimation

A system and method for automatic rigging of three dimensional characters for facial animation provide a rigged mesh for an original three dimensional mesh. A representative mesh is generated from the original mesh. Segments, key points, a bone set, and skinning weights are then determined for the representative mesh. The Skinning weights and bone set are placed in the original mesh to generate the rigged mesh.

Automatic rigging of three dimensional characters for animation

The present disclosure is directed toward systems and methods for predicting lighting conditions. In particular, the systems and methods described herein analyze a single low-dynamic range digital image to estimate a set of high-dynamic range lighting conditions associated with the single low-dynamic range lighting digital image. Additionally, the systems and methods described herein train a convolutional neural network to extrapolate lighting conditions from a digital image. The systems and methods also augment low-dynamic range information from the single low-dynamic range digital image by using a sky model algorithm to predict high-dynamic range lighting conditions.

Estimating HDR lighting conditions from a single LDR digital image

Embodiments of the present invention provide systems, methods, and computer storage media directed to generating training image data for a convolutional neural network, encoding parameters into a convolutional neural network, and employing a convolutional neural network that estimates camera calibration parameters of a camera responsible for capturing a given digital image. A plurality of different digital images can be extracted from a single panoramic image given a range of camera calibration parameters that correspond to a determined range of plausible camera calibration parameters. With each digital image in the plurality of extracted different digital images having a corresponding set of known camera calibration parameters, the digital images can be provided to the convolutional neural network to establish high-confidence correlations between detectable characteristics of a digital image and its corresponding set of camera calibration parameters. Once trained, the convolutional neural network can receive a new digital image, and based on detected image characteristics thereof, estimate a corresponding set of camera calibration parameters with a calculated level of confidence.

Neural network-based camera calibration

Methods and systems are provided for determining high-dynamic range lighting parameters for input low-dynamic range images. A neural network system can be trained to estimate lighting parameters for input images where the input images are synthetic and real low-dynamic range images. Such a neural network system can be trained using differences between a simple scene rendered using the estimated lighting parameters and the same simple scene rendered using known ground-truth lighting parameters. Such a neural network system can also be trained such that the synthetic and real low-dynamic range images are mapped in roughly the same distribution. Such a trained neural network system can be used to input a low-dynamic range image determine high-dynamic range lighting parameters.

Learning to estimate high-dynamic range outdoor lighting parameters

System and method for automatic rigging of three dimensional characters for facial animation

Extrapolating lighting conditions from a single digital image

Methods and systems are provided for using a single image of an indoor scene to estimate illumination of an environment that includes the portion captured in the image. A neural network system may be trained to estimate illumination by generating recovery light masks indicating a probability of each pixel within the larger environment being a light source. Additionally, low-frequency RGB images may be generated that indicating low-frequency information for the environment. The neural network system may be trained using training input images that are extracted from known panoramic images. Once trained, the neural network system infers plausible illumination information from a single image to realistically illumination images and objects being manipulated in graphics applications, such as with image compositing, modeling, and reconstruction.

Illumination estimation from a single image

Systems and methods for generating recommendations for animations to apply to animate 3D characters in accordance with embodiments of the invention are disclosed. One embodiment includes an animation server and a database containing metadata describing a plurality of animations and the compatibility of ordered pairs of the described animations. In addition, the animation server is configured to receive requests for animation recommendations identifying a first animation, generate a recommendation of at least one animation described in the database based upon the first animation, receive a selection of an animation described in the database, and concatenate at least the first animation and the selected animation.

Generating smooth animation sequences

Certain embodiments involve refining local parameterizations that apply two-dimensional ('2D') images to three-dimensional ('3D') models. For instance, a particular parameterization-initialization process is select based on one or more features of a target mesh region. An initial local parameterization for a 2D image is generated from this parameterization-initialization process. A quality metric for the initial local parameterization is computed, and the local parameterization is modified to improve the quality metric. The 3D model is modified by applying image points from the 2D image to the target mesh region in accordance with the modified local parameterization.

Refining local parameterizations for applying two-dimensional images to three-dimensional models

Growing community of inventors

San Francisco, CA, United States of America

Emiliano Gambaretto