Meta Platforms Technologies, LLC

Yuichi Taguchi

Fabian Langguth

Anush Mohan

Alexandru-Eugen Ichim

Gaurav Chaurasia

Manuel Werlberger

Boyang Zhang

Shir Lene Lim

Wei Lyu

Yifu Zhang

A method of interacting with an artificial-reality (AR) content at an AR headset that includes cameras and displays is described. The method includes, while the AR headset has a first position within a physical environment that includes an object, if a distance of the object is within a threshold collision distance from the AR headset, presenting a meshed representation of the object. The meshed representation is displayed at first respective locations on the displays such that the meshed representation is viewable within the artificial reality. After the AR headset moves to a second position with a different distance, and if the different distance is within the threshold collision distance from the AR headset, moving the meshed representation of the object to second respective locations within the artificial reality. The second respective locations correspond to the position of the object in the physical environment.

Presenting meshed representations of physical objects within defined boundaries for interacting with artificial-reality content, and systems and methods of use thereof

Implementations described herein can share scene data between multiple artificial reality (XR) devices, such as head-mounted displays (HMDs). A first XR device can retrieve stored spatial anchors associated with a scene, and align a coordinate frame with coordinate frames for a second XR device using the spatial anchors. The second XR device can have identified physical objects in the scene with corresponding object types defined in a scene lexicon (e.g., with a camera on the second XR device capturing images of physical objects and performing object recognition or through manual tagging of objects with object types), and store the object types and locations of the physical objects as scene data. The first XR device can obtain the scene data based on associations between the scene data and the spatial anchors and render virtual objects using the scene data, without having to rescan the scene for the physical objects.

Shared scene co-location for artificial reality devices

In one embodiment, a method includes receiving sensor data of a scene captured using one or more sensors, generating (1) a number of virtual surfaces representing a number of detected planar surfaces in the scene and (2) a point cloud representing detected features of objects in the scene based on the sensor data, assigning each point in the point cloud to one or more of the number of virtual surfaces, generating occupancy volumes for each of the number of virtual surfaces based on the points assigned to the virtual surface, generating a datastore including the number of virtual surfaces, the occupancy volumes of each of the number of virtual surfaces, and a spatial relationship between the number of virtual surfaces, receiving a query, and sending a response to the query, the response including an identified subset of the plurality of virtual surfaces in the datastore that satisfy the query.

Environment model with surfaces and per-surface volumes

Growing community of inventors

Los Gatos, CA, United States of America

Yuichi Taguchi