Meta Platforms Technologies, LLC

Facebook, Inc.

Facebook Technologies, LLC

Meta Platforms, Inc.

Boyang Zhang

Chengyuan Yan

Adrian Brian Ratter

Paul Austin Buckley

Jack Hood Profit, Jr

Joel Hesch

Alexandru-Eugen Ichim

Gaurav Chaurasia

Manuel Werlberger

Yuichi Taguchi

Yifu Zhang

A method of interacting with an artificial-reality (AR) content at an AR headset that includes cameras and displays is described. The method includes, while the AR headset has a first position within a physical environment that includes an object, if a distance of the object is within a threshold collision distance from the AR headset, presenting a meshed representation of the object. The meshed representation is displayed at first respective locations on the displays such that the meshed representation is viewable within the artificial reality. After the AR headset moves to a second position with a different distance, and if the different distance is within the threshold collision distance from the AR headset, moving the meshed representation of the object to second respective locations within the artificial reality. The second respective locations correspond to the position of the object in the physical environment.

Presenting meshed representations of physical objects within defined boundaries for interacting with artificial-reality content, and systems and methods of use thereof

A platform system receives sensor data describing the state and orientation of a tracked object and models the pose of the tracked object to determine user interactions with the platform system. To ensure that incorrect sensor data due to a saturation event or connection loss does not impact user experience, the platform system identifies regions for correction in sensor data streams based on the sensor data being at or above a saturation limit or not being received. The platform system predicts sensor data for an identified region of correction by applying a fit corresponding to points adjacent to the region for correction and determining predicted sensor data using the applied fit. The predicted sensor data is used to correct the modeled pose for the tracked object.

Modeling poses of tracked objects by predicting sensor data

In one embodiment, a method includes receiving, from a controller, a data packet including (1) a plurality of samples each corresponding to measurements from a motion sensor and (2) a timestamp corresponding to a measurement time of one of the samples as measured by a clock of the controller; determining, based on the timestamp, an estimated measurement time relative to a local clock for each of the plurality of samples that is not associated with the timestamp; and converting each of the timestamp and the estimated measurement times to a corresponding synchronization time using a learned relationship relating the clock of the controller and the local clock. The learned relationship is iteratively learned based on previously received data packets from the controller. The synchronization time associated with each of the plurality of samples represents an estimated time, relative to the local clock, at which the sample was measured.

Headset clock synchronization

In one embodiment, a method includes receiving motion data from a motion sensor during a packet-transmission interval of a wireless protocol. The motion data corresponds to a first pre-determined number of samples measured at a first sampling frequency. Each sample is associated with a first timestamp corresponding to a measurement time of that sample during the packet-transmission interval. The method also includes converting the motion data to correspond to a second pre-determined number of samples. The second pre-determined number is fewer than the first pre-determined number. The method also includes determining a second timestamp for each of the second pre-determined number of samples. The second timestamps are within the packet-transmission interval and represent measurement times at a second sampling frequency that is lower than the first sampling frequency. The method also includes combining the converted motion data and the corresponding second timestamps into a first data packet.

Sensor data compression

Growing community of inventors

Cupertino, CA, United States of America

Boyang Zhang