Nvidia Corporation

Deluxe Media Creative Services Inc.

Simon Yuen

Rev Lebaredian

Ratin Kumar

Jonathan Cohen

Santanu Dutta

Dmitry Korobchenko

Evgeny Aleksandrovich Tumanov

Peter Shipkov

Kevin Margo

Malte Wagener

In various examples, animations may be generated using audio-driven body animation synthesized with voice tempo. For example, full body animation may be driven from an audio input representative of recorded speech, where voice tempo (e.g., a number of phonemes per unit time) may be used to generate a 1D audio signal for comparing to datasets including data samples that each include an animation and a corresponding 1D audio signal. One or more loss functions may be used to compare the 1D audio signal from the input audio to the audio signals of the datasets, as well as to compare joint information of joints of an actor between animations of two or more data samples, in order to identify optimal transition points between the animations. The animations may then be stitched together—e.g., using interpolation and/or a neural network trained to seamlessly stitch sequences together—using the transition points.

Synthetic audio-driven body animation using voice tempo

In various examples, a virtually animated and interactive agent may be rendered for visual and audible communication with one or more users with an application. For example, a conversational artificial intelligence (AI) assistant may be rendered and displayed for visual communication in addition to audible communication with end-users. As such, the AI assistant may leverage the visual domain—in addition to the audible domain—to more clearly communicate with users, including interacting with a virtual environment in which the AI assistant is rendered. Similarly, the AI assistant may leverage audio, video, and/or text inputs from a user to determine a request, mood, gesture, and/or posture of a user for more accurately responding to and interacting with the user.

Conversational AI platform with rendered graphical output

Various embodiments of systems and methods for deforming a virtual geometric object are disclosed. In accordance with at least one embodiment, a method for deforming a virtual geometric object substantially real-time includes the operations of relaxing a hi-res mesh defining a shape of the object, wherein the hi-res mesh corresponds to a proxy mesh, relaxing the hi-res mesh at least once, recording the location of the elements of the hi-res mesh as relaxed, deforming the proxy mesh, adjusting the location of one or more elements on the hi-res mesh relative to the as deformed proxy mesh, relaxing the adjusted elements on the deformed hi-res mesh, and adjusting the location of the hi-res mesh as deformed in view of the previously recorded elements of the hi-res mesh as relaxed both pre and after deformation.

Systems and methods for automated virtual geometry deformation

Growing community of inventors

Playa Vista, CA, United States of America

Simon Yuen