Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Bryan Michael Anenberg

Elif Albuz

Ronit Kassis

Peihong Guo

Samuel Alan Johnson

Xingze He

Colin Lea

Nikita Shulga

Examples described herein include systems for reconstructing facial image data from partial frame data and landmark data. Systems for generating the partial frame data and landmark data are described. Neural networks may be used to reconstruct the facial image data and/or generate the partial frame data. In this manner, compression of facial image data may be achieved in some examples.

Facial image data generation using partial frame data and landmark data

In one embodiment, a method includes receiving an audio signal comprising a plurality of speech units, processing the audio signal to associate each of the speech units with a corresponding lip animation, determining pitch information associated with each of the plurality of speech units, processing the pitch information of each of the plurality of speech units to associate at least one of the speech units with a facial-component animation, and presenting the audio signal with a displayed animation of a face, wherein the animation of the face displays the lip animation associated with each of the speech units and the facial-component animation associated with the at least one speech unit. The animation of the face may be displayed in real time with the audio signal. The facial component animation may include animation of the lips, eyebrows, eyelids, and other portion of the upper face.

Audio-based face tracking and lip syncing for natural facial animation and lip movement

Growing community of inventors

Mountain View, CA, United States of America

Bryan Michael Anenberg