Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Sheng Shen

Alexander Sorkine Hornung

Gioacchino Noris

Alessia Marra

Jennifer Lynn Spurlock

Jeffrey Witthuhn

William Arthur Hugh Steptoe

Sarthak Ray

Fangwei Lee

Björn Wanbo

Derrick Readinger

Jonathan Mallinson

Connor Treacy

Larissa Heike Laich

Javier Alejandro Sierra Santos

Federico Ruiz

Manuel Ricardo Freire Santos

Marcus Tanner

Aleksejs Anpilogovs

Panya Inversin

Jeffrey Wai-King Lock

Jonathan Richard Mayoh

Michael James LeBeau

Chengyuan Yan

Joseph Davis Greer

Willy Huang

Doruk Senkal

Szeyin Lee

Michael James Lebeau

Anurag Sharma

The various implementations described herein include methods and systems for providing input capabilities at various fidelity levels. In one aspect, a method includes receiving, from an application, a request identifying an input capability for making an input operation available within the application. The method further includes, in response to receiving the request: identifying techniques that the artificial-reality system can use to make the requested input capability available to the application using data from one or more devices; selecting a first technique for making the requested input capability available to the application; and using the first technique to provide to the application data to allow for performance of the requested input capability.

Hardware-agnostic input framework for providing input capabilities at various fidelity levels, and systems and methods of use thereof

Methods of interpreting in-air hand gestures based on orientations of a wrist-wearable device are provided. The method includes receiving, from sensors of a wrist-wearable device, data associated with performance of an in-air hand gesture during a first period of time. The method includes, when determining the data indicates that the wrist-wearable device had a first orientation when the in-air hand gesture was performed during the first period of time, causing performance of a first operation. The method includes receiving, from sensors of the wrist-wearable device, new data associated with performance of the in-air hand gesture during a second period of time, the second period of time being after the first period of time. The method includes, when determining that the new data indicates that the wrist-wearable device had a second orientation when the in-air hand gesture was performed during the second period of time, causing performance of a second operation.

Systems for interpreting thumb movements of in-air hand gestures for controlling user interfaces based on spatial orientations of a user's hand, and method of use thereof

A system having at least a first component and a second component positioned at different locations on a user's body (e.g., on the user's head and held on the user's hand). Each component includes at least one inertial measurement unit (IMU) configured generate measurements indicating acceleration and angular rate data. The generated measurements of the IMUs are used with ground truth information indicating the positions of the first and second component to generate a set of training data to train a neural network configured to predict a relative position between the first and second components based on IMU measurements received over a predetermined time period. Because the neural network is trained based upon movements of a human user, the neural network model takes into account physiological constraints of the user in determining how the set of potential positions of the different components may change over time, reducing potential error.

Controller position tracking using inertial measurement units and machine learning

Aspects of the present disclosure are directed to creating and administering artificial reality collaborative working environments and providing interaction modes for them. An XR work system can provide and control such artificial reality collaborative working environments to enable, for example, A) links between real-world surfaces and XR surfaces; B) links between multiple real-world areas to XR areas with dedicated functionality; C) maintaining access, while inside the artificial reality working environment, to real-world work tools such as the user's computer screen and keyboard; D) various hand and controller modes for different interaction and collaboration modalities; E) use-based, multi-desk collaborative room configurations; and F) context-based auto population of users and content items into the artificial reality working environment.

Artificial reality collaborative working environments

Growing community of inventors

Seattle, WA, United States of America

Sheng Shen