Loomai, Inc.

Roblox Corporation

Will Welch

Kiran Bhat

Vivek Verma

Akash Garg

Michael Daniel Flynn

Ian Sachs

Inaki Navarro

Tijmen Verhulsdonck

Dario Kneubuhler

Eloi Du Bois

Chong Shang

Dominic Monn

Eloi Henri Homere Du Bois

Senthil Radhakrishnan

Rishabh Battulwar

Implementations described herein relate to methods, systems, and computer-readable media to generate animations for a 3D avatar from input video captured at a client device. A camera may capture video of a face while a trained face detection model and a trained regression model output a set of FACS weights, head poses, and facial landmarks to be translated into the animations of the 3D avatar. Additionally, a higher level-of-detail may be intelligently selected based upon user preferences and/or computing conditions at the client device.

Robust facial animation from video using neural networks

Systems and methods for animating from audio in accordance with embodiments of the invention are illustrated. One embodiment includes a method for generating animation from audio. The method includes steps for receiving input audio data, generating an embedding for the input audio data, and generating several predictions for several tasks from the generated embedding. The several predictions includes at least one of blendshape weights, event detection, and/or voice activity detection. The method includes steps for generating a final prediction from the several predictions, where the final prediction includes a set of blendshape weights, and generating an output based on the generated final prediction.

Systems and methods for animation generation

System and methods for computer animations of 3D models of heads generated from images of faces is disclosed. A 2D captured image that includes an image of a face can be received and used to generate a static 3D model of a head. A rig can be fit to the static 3D model to generate an animation-ready 3D generative model. Sets of rigs can be parameters that each map to particular sounds or particular facial movement observed in a video. These mappings can be used to generate a playlists of sets of rig parameters based upon received audio or video content. The playlist may be played in synchronization with an audio rendition of the audio content. Methods can receive a captured image, identify taxonomy attributes from the captured image, select a template model for the captured image, and perform a shape solve for the selected template model based on the identified taxonomy attributes.

Systems and methods for generating computer ready animation models of a human head from captured data images

System and methods for computer animations of 3D models of heads generated from images of faces is disclosed. A 2D captured image that includes an image of a face can be received and used to generate a static 3D model of a head. A rig can be fit to the static 3D model to generate an animation-ready 3D generative model. Sets of rigs can be parameters that each map to particular sounds. These mappings can be used to generate a playlists of sets of rig parameters based upon received audio content. The playlist may be played in synchronization with an audio rendition of the audio content.

Systems and methods for animating models from audio data

Growing community of inventors

San Francisco, CA, United States of America

Will Welch