Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Benjamin Antoine Georges Lefaudeux

Samuel Alan Johnson

Mahdi Salmani Rahimi

Shaik Shabnam Nizamudeen Basha

Carsten Sebastian Stoll

Kishore Venkateshan

Weiguang Si

Robert Foster

Kishore Venkatehsan

A virtual space configuration system of an artificial reality system can detect a user posture and provide various corresponding customizations of the system's virtual space. The virtual space configuration system can, when a user is in a seated posture, provide for seated virtual space customizations. In various implementations, these customizations can include allowing adjustment of a floor height; setting a flag that can be surfaced to applications to adjust the applications' mechanics for seated users; customizing display of virtual space boundaries when in seated mode to be less intrusive; providing options to detect when a user leaves seated mode and trigger corresponding actions; provide a passthrough workspace area allowing a user to interact with certain real-world objects naturally without having to remove a virtual reality headset; or automatically determining virtual space dimensions for seated users.

Posture-based virtual space configurations

A computing system may receive sensor data from one or more sensors coupled to a user. Based on sensor data, the computing system may generate an upper body pose that corresponds to a first portion of a body of the user, which comprises a head and an arm of the user. The computing system may determine contextual information associated with the user. The computing system may generate a lower body pose corresponding to a second portion of the body of the user comprising a leg of the user based on the upper body pose and the contextual information. The computing system may generate a full body pose of the user based on the first upper body pose and the lower body pose.

Systems and methods for predicting lower body poses

A computing system may receive an image captured by a camera, and determine based on (1) one or more regions in the image that include a portion of a body of a user, and (2) a camera pose of the camera, a three-dimensional volume constraint that stems from the camera pose in which an elbow of the user is likely to be located. The system may infer, based on at least the three-dimensional volume constraint, an elbow pose of the user, wherein the elbow pose includes a location associated with an elbow of the user in three-dimensional space. The system may determine the location associated with the elbow is outside the three-dimensional volume constraint, and in response to the determination, adjust the inferred elbow pose. The system may generate, based on the adjusted elbow pose, a body pose of the user that includes at least the adjusted elbow pose.

Systems and methods for predicting elbow joint poses

A computing system may receive sensor data from one or more sensors coupled to a user. Based on sensor data, the computing system may generate a first upper body pose that corresponds to a first portion of a body of the user, which comprises a head and an arm of the user. The computing system may process the upper body pose of the user to generate a temporal sequence of lower body poses comprising a first lower body pose associated with the first time and a second lower body pose associated with a second time, which may correspond to a second portion of the body of the user comprising a leg of the user. The computing system may generate one or more full body poses of the user based on at least the first upper body pose and the temporal sequence of lower body poses.

A computing system may determine a head pose and a wrist pose of a user based on sensor data captured by a head-mounted device. The computing system may further receive an image captured by a camera of the head-mounted device that includes at least portion of the body of the user. Using this image, the computing system may determine one or more regions in the image that correspond to the body of the user, and determine, based on these regions and a camera pose of the camera, a three-dimensional volume constraint in which an elbow of the user is likely to be located when the image was captured. From this information, the computing system may utilize the head pose, the wrist pose, and the three-dimensional volume constraint to infer a body pose of the user that includes at least an inferred elbow pose associated with the user.

A computing system may receive sensor data from one or more sensors coupled to a user. Based on this sensor data, the computing system may generate an upper body pose that corresponds to a first portion of a body of the user, which may comprise a head and an arm of the user. The computing system may process the upper body pose of the user using a machine learning model to generate a lower body pose that corresponds to a second portion of the body of the user, which may comprise a leg of the user. The computing system may generate a full body pose of the user based on the upper body pose and the lower body pose.

Artificial reality (AR) systems track pose and skeletal positioning for multiple co-located participants, each having a head mounted display (HMD). Participants can join a shared artificial reality event or experience with others in the same location. Each participant's HMD can independently render AR content for the participant based on the participant's pose and pose information obtained from other participants' HMDs. A participating HMD may broadcast tracking estimates for skeletal points of interest (e.g., joints, finger tips, knees, ankle points, etc.) that are within the field-of-view of the HMD's cameras and/or sensors. A participating HMD may receive skeletal position information determined by other HMDs, and aggregate the received tracking information along with internal tracking information to construct an accurate, full estimate of its own pose and skeletal positioning information for its corresponding participant.

Co-located pose estimation in a shared artificial reality environment

An artificial reality system includes a head mounted display (HMD) and a hand-held device. A virtual boundary can provide for user safety during an artificial reality session. For example, the virtual boundary may be defined to protect the user from collisions with physical objects, such as walls, furniture, or the like, while the user is immersed in the artificial reality session. The artificial reality system can constrain the predicted motion path of a portion of a user's body based on body configuration parameters and range of motion of the joints associated with the body portion. The artificial reality system can then use the constrained motion paths to determine whether or not the virtual boundary warning should be activated.

Growing community of inventors

Menlo Park, CA, United States of America

Benjamin Antoine Georges Lefaudeux