Disney Enterprises, Inc.

Meta Platforms Technologies, LLC

Nvidia Corporation

Eth Zurich

Ye Yuan

Robert Walker Sumner

Stelian Coros

Jason Saragih

Tomas Simon Kreuz

Shih-En Wei

Martin Guay

Kris Makoto Kitani

Jan Kautz

Pavlo Molchanov

Umar Iqbal

Dominik Tobias Borer

Moritz Geilinger

Gökçen Çimen

Simone Guggiari

A method for simulating a solid body animation of a subject includes retrieving a first frame that includes a body image of a subject. The method also includes selecting, from the first frame, multiple key points within the body image of the subject that define a hull of a body part and multiple joint points that define a joint between two body parts, identifying a geometry, a speed, and a mass of the body part to include in a dynamic model of the subject, based on the key points and the joint points, determining, based on the dynamic model of the subject, a pose of the subject in a second frame after the first frame in a video stream, and providing the video stream to an immersive reality application running on a client device.

Simulated control for 3-dimensional human poses in virtual reality environments

In order to determine accurate three-dimensional (3D) models for objects within a video, the objects are first identified and tracked within the video, and a pose and shape are estimated for these tracked objects. A translation and global orientation are removed from the tracked objects to determine local motion for the objects, and motion infilling is performed to fill in any missing portions for the object within the video. A global trajectory is then determined for the objects within the video, and the infilled motion and global trajectory are then used to determine infilled global motion for the object within the video. This enables the accurate depiction of each object as a 3D pose sequence for that model that accounts for occlusions and global factors within the video.

Performing occlusion-aware global 3D pose and shape estimation of articulated objects

Techniques for generating locomotion data for animating a virtual quadruped model, starting at an origin point and travelling to a destination point along a defined path. A virtual skeletal structure for the virtual quadruped model is analyzed to identify a torso region, limbs each ending in a respective end effector, and limb attributes. A predefined locomotion template for virtual quadruped characters is retrieved and mapped to the virtual quadruped model by aligning the torso region and the plurality of limbs of the virtual skeletal structure for the virtual quadruped with a second torso region and a second plurality of limbs of the predefined locomotion template. Locomotion data is generated for the virtual quadruped model based on the defined path and by upscaling the mapped predefined locomotion template, based at least on the set of limb attributes determined by analyzing the virtual skeletal structure for the virtual quadruped model.

Automatically generating quadruped locomotion controllers

A simulation engine models interactions between a simulated character and real-world objects to produce a physically realistic augmented reality (AR) simulation. The simulation engine recognizes a given real-world object and then identifies, within a library of object models, an object model corresponding to that object. The simulation engine projects the object model onto the real-world object such that the object model is geometrically aligned with the real-world object. When the simulated character encounters the real-world object, the simulation engine models interactions between the simulated character and the real-world object by adjusting the kinematics of the simulated character relative to the object model associated with the real-world object.

Modeling interactions between simulated characters and real-world objects for more realistic augmented reality

Growing community of inventors

Pittsburgh, PA, United States of America

Ye Yuan