Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Muzaffer Kal

Richard Andrew Newcombe

Vincent Lee

Armin Alaghi

Andrew Samuel Berkovich

Shlomo Alkalay

Anastasios Mourikis

Amr Suleiman

Muhammad Huzaifa

A method includes accessing map data of an area of a real environment, the map data comprising three-dimensional feature descriptors describing features visible in the real environment. A plurality of map packages are generated based on the map data, wherein each of the map packages (1) corresponds to a two-dimensional sub-area within the area of the real environment, and (2) comprises a subset of the three-dimensional feature descriptors describing features visible in the sub-area. A first sequence of the plurality of map packages are broadcast through one or more base stations, wherein the first sequence is based on the two-dimensional sub-area of each of the map packages, wherein each of the map packages is configured to be received and used by an artificial-reality device to determine a pose of the artificial-reality device in the associated sub-area based on the associated subset of the three-dimensional feature descriptors.

Methods and systems to facilitate passive relocalization using three-dimensional maps

In one embodiment, a computing system accesses a set of 3D locations associated with features in an environment previously captured by a camera from a previous camera pose. The computing system determines a predicted camera pose using the previous camera pose and motion measurements generated using a motion sensor associated with the camera. The computing system projects the set of 3D locations toward the predicted camera pose and onto a 2D image plane associated with the camera. The computing system generates, based on the projected set of 3D locations on the 2D image plane, an activation map specifying a subset of the pixel sensors of the camera that are to be activated. The computing system instructs, using the activation map, the camera to activate the subset of pixel sensors to capture a new image of the environment. The computing system reads pixel values of the new image.

Visual inertial odometry localization using sparse sensors

In one embodiment, a method for tracking includes capturing a first frame of the environment using a first camera, identifying, in the first frame, a first patch that corresponds to the first feature, accessing a first local memory of the first camera that stores reference patches identified in one or more previous frames captured by the first camera, and determining that none of the reference patches stored in the first local memory corresponds to the first feature. The method further includes receiving, from a second camera through a data link connecting the second camera with the first camera, a reference patch corresponding to the first feature. The reference patch is identified in a previous frame captured by the second camera and of the second camera. The method may then determine correspondence data between the first patch and the reference patch, and tracks the first feature in the environment based on the determined correspondence data.

Distributed sensor module for tracking

Growing community of inventors

Redmond, WA, United States of America

Muzaffer Kal