Meta Platforms Technologies, LLC

Kishore Venkateshan

Samuel Alan Johnson

Benjamin Antoine Georges Lefaudeux

Mahdi Salmani Rahimi

Carsten Sebastian Stoll

A computing system may receive sensor data from one or more sensors coupled to a user. Based on sensor data, the computing system may generate an upper body pose that corresponds to a first portion of a body of the user, which comprises a head and an arm of the user. The computing system may determine contextual information associated with the user. The computing system may generate a lower body pose corresponding to a second portion of the body of the user comprising a leg of the user based on the upper body pose and the contextual information. The computing system may generate a full body pose of the user based on the first upper body pose and the lower body pose.

Systems and methods for predicting lower body poses

A computing system may receive an image captured by a camera, and determine based on (1) one or more regions in the image that include a portion of a body of a user, and (2) a camera pose of the camera, a three-dimensional volume constraint that stems from the camera pose in which an elbow of the user is likely to be located. The system may infer, based on at least the three-dimensional volume constraint, an elbow pose of the user, wherein the elbow pose includes a location associated with an elbow of the user in three-dimensional space. The system may determine the location associated with the elbow is outside the three-dimensional volume constraint, and in response to the determination, adjust the inferred elbow pose. The system may generate, based on the adjusted elbow pose, a body pose of the user that includes at least the adjusted elbow pose.

Systems and methods for predicting elbow joint poses

A computing system may receive sensor data from one or more sensors coupled to a user. Based on sensor data, the computing system may generate a first upper body pose that corresponds to a first portion of a body of the user, which comprises a head and an arm of the user. The computing system may process the upper body pose of the user to generate a temporal sequence of lower body poses comprising a first lower body pose associated with the first time and a second lower body pose associated with a second time, which may correspond to a second portion of the body of the user comprising a leg of the user. The computing system may generate one or more full body poses of the user based on at least the first upper body pose and the temporal sequence of lower body poses.

A computing system may determine a head pose and a wrist pose of a user based on sensor data captured by a head-mounted device. The computing system may further receive an image captured by a camera of the head-mounted device that includes at least portion of the body of the user. Using this image, the computing system may determine one or more regions in the image that correspond to the body of the user, and determine, based on these regions and a camera pose of the camera, a three-dimensional volume constraint in which an elbow of the user is likely to be located when the image was captured. From this information, the computing system may utilize the head pose, the wrist pose, and the three-dimensional volume constraint to infer a body pose of the user that includes at least an inferred elbow pose associated with the user.

Growing community of inventors

Menlo Park, CA, United States of America

Kishore Venkateshan