Adobe, Inc.

Mixamo, Inc.

Leland Stanford Junior University

Facebook, Inc.

Politecnico Di Milano

Stefano Koenig Corazza

Emiliano Gambaretto

Edilson De Aguiar

Daniel Babcock

Sylvio Herve Drouin

Yaniv De Ridder

Nazim Kareemi

Charles Pina

Prasanna Vasudevan

Graham Taylor

Lars Mündermann

Lee Brimelow

Erwan Maigret

David Montero

Kalyan Krishna Sunkavalli

Gavin Stuart Peter Miller

Yannick Hold-Geoffroy

Thomas P Andriacchi

Milos Hasan

Zexiang Xu

Thomas P Andrlacchl

Emillano Gambaretto

Yu-Ying Yeh

Charles Piña

The present disclosure relates to an AR animation generation system that detects a change in position of a mobile computing system in a real-world environment, determines that a position for a virtual object in an augmented reality (AR) scene is to be changed from a first position in the AR scene to a second position in the AR scene, identifies an animation profile to be used for animating the virtual object, wherein the animation profile is associated with the virtual object, and animates the virtual object in the AR scene using the animation profile. Animating the virtual object in the AR scene includes moving the virtual object in the AR scene from the first position to the second position along a path, wherein the path and a movement of the virtual object along the path are determined based on the animation profile.

Generating animations in an augmented reality environment

Embodiments of the technology described herein, make unknown material-maps in a Physically Based Rendering (PBR) asset usable through an identification process that relies, at least in part, on image analysis. In addition, when a desired material-map type is completely missing from a PBR asset the technology described herein may generate a suitable synthetic material map for use in rendering. In one aspect, the correct map type is assigned using a machine classifier, such as a convolutional neural network, which analyzes image content of the unknown material map and produce a classification. The technology described herein also correlates material maps into material definitions using a combination of the material-map type and similarity analysis. The technology described herein may generate synthetic maps to be used in place of the missing material maps. The synthetic maps may be generated using a Generative Adversarial Network (GAN).

Material map identification and augmentation

A system and method for automatic rigging of three dimensional characters for facial animation provide a rigged mesh for an original three dimensional mesh. A representative mesh is generated from the original mesh. Segments, key points, a bone set, and skinning weights are then determined for the representative mesh. The Skinning weights and bone set are placed in the original mesh to generate the rigged mesh.

Automatic rigging of three dimensional characters for animation

The present disclosure relates to an AR animation generation system identifies an animation profile for animating the virtual object displayed in an augmented reality (AR) scene. The AR animation generation system creates a link between the virtual object and the mobile computing system based upon a position of the virtual object within the AR scene and a position of a mobile device in a real-world environment. The link enables determining for each position of the mobile device in the real-world environment, a corresponding position for the virtual object in the AR scene. The AR animation generation system animates the virtual object using the mobile device by detecting a change in position in the real-world environment of the mobile computing system from a first position to a second position and using the link to determine a change in position for the virtual object in the AR scene from a first position to a second position, The AR animation generation system updates the AR scene to display the virtual object in the second position in the AR scene.

System and method for automatic rigging of three dimensional characters for facial animation

Three dimensional digital content interaction and control techniques are described. A digital medium environment is configured to protect against unauthorized access to three-dimensional digital content. One or more inputs are received involving user selection of one or more parameters of user interactive features to change an appearance of the three-dimensional digital content. A plurality of images is rendered from the three-dimensional digital content. The rendered plurality of images has a difference in the one or more parameters, one to another. A control is associated that supports user interaction to navigate through the plurality of images based on the difference in the one or more parameters. Distribution is controlled of the plurality of images and the associated control for consumption including navigation through the plurality of images using the associated control for the one or more parameters of the user interactive features.

3D digital content interaction and control

Systems and methods for generating recommendations for animations to apply to animate 3D characters in accordance with embodiments of the invention are disclosed. One embodiment includes an animation server and a database containing metadata describing a plurality of animations and the compatibility of ordered pairs of the described animations. In addition, the animation server is configured to receive requests for animation recommendations identifying a first animation, generate a recommendation of at least one animation described in the database based upon the first animation, receive a selection of an animation described in the database, and concatenate at least the first animation and the selected animation.

Generating smooth animation sequences

A digital medium environment includes at least one computing device (e.g., a user's mobile device) in an environment. Systems and techniques are described for presenting a photorealistic augmented reality experience in real time on the computing device. A lighting representation of the environment is obtained from a partial view of the environment, such as from an image of the environment, and used to render a 3D asset (e.g., graphics of a synthetic object). The 3D asset is inserted into the environment by compositing the rendered 3D asset with the image of the environment accounting for occlusion and based on the lighting representation to form a composite image. Photorealistic processing is applied to the composite image by adding lighting for the rendered 3D asset to a portion of the background of the composite image. Consequently, the inserted graphic is indistinguishable from the real-world scene.

Photorealistic augmented reality system

Systems and methods for animation recommendations

Systems and methods for automatically generating animation-ready 3D character models based upon model parameter, clothing selections, and texture-region color component selections are described. One embodiment of the invention includes an application server configured to receive the user defined model parameters and the at least one texture selection via a user interface. In addition, the application server includes a generative model and the application server is configured to generate a 3D mesh based upon the user defined model parameters using the generative model and to apply texture to the generated mesh based upon the at least one texture selection.

Real time concurrent design of shape, texture, and motion for 3D character animation

The present disclosure is directed to integrating external 3D models into a character creation system. In general, a character creation system imports an external 3D model by determining correspondence values for each vertex within the 3D model. Once imported, a user can customize the 3D character by adding texture to the character, adjusting character features, swapping out one or more character features, adding clothes and accessories to the character, automatically rigging the character, and/or animating the character.

Automatically determining correspondences between three-dimensional models

Systems and methods in accordance with embodiments of the invention enable collaborative creation, transmission, sharing, non-linear exploration, and modification of animated video messages. One embodiment includes a video camera, a processor, a network interface, and storage containing an animated message application, and a 3D character model. In addition, the animated message application configures the processor to: capture a video sequence using the video camera; detect a human face within a sequence of video frames; track changes in human facial expression of a human face detected within a sequence of video frames; map tracked changes in human facial expression to motion data, where the motion data is generated to animate the 3D character model; apply motion data to animate the 3D character model; render an animation of the 3D character model into a file as encoded video; and transmit the encoded video to a remote device via the network interface.

Systems and methods for creating and distributing modifiable animated video messages

Systems and methods for creating animations using human faces

Systems and methods are described for animating 3D characters using synthetic motion data generated by motion models in response to a high level description of a desired sequence of motion provided by an animator. In a number of embodiments, the synthetic motion data is streamed to a user device that includes a rendering engine and the user device renders an animation of a 3D character using the streamed synthetic motion data. In several embodiments, an animator can upload a custom model of a 3D character or a custom 3D character is generated by the server system in response to a high level description of a desired 3D character provided by the user and the synthetic motion data generated by the generative model is retargeted to animate the custom 3D character.

Web platform for interactive design, synthesis and delivery of 3D character motion data

Systems and methods are described for performing spatial and temporal compression of deformable mesh based representations of 3D character motion allowing the visualization of high-resolution 3D character animations in real time. In a number of embodiments, the deformable mesh based representation of the 3D character motion is used to automatically generate an interconnected graph based representation of the same 3D character motion. The interconnected graph based representation can include an interconnected graph that is used to drive mesh clusters during the rendering of a 3D character animation. The interconnected graph based representation provides spatial compression of the deformable mesh based representation, and further compression can be achieved by applying temporal compression processes to the time-varying behavior of the mesh clusters. Even greater compression can be achieved by eliminating redundant data from the file format containing the interconnected graph based representation of the 3D character motion that would otherwise be repeatedly provided to a game engine during rendering, and by applying loss-less data compression to the data of the file itself.

Growing community of inventors

Richmond, CA, United States of America

Stefano Koenig Corazza