Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Meta Platforms, Inc.

Fabian Langguth

Alexander Sorkine Hornung

Yuichi Taguchi

Richard S Szeliski

Alessia Marra

Johannes Peter Kopf

Matthew James Alderman

Andrei Chtcherbatchenko

Suhib Fakhri Mahmod Alsisan

Bo Yin

Shi Chen

Francis Yunfeng Ge

In one embodiment, a computing system may monitor one or more regions of a first 3D model of an environment to determine a frequency the respective region changes. The computing system may determine a first region of the one or more regions is static based on the frequency the respective region changes. The computing system may delay a first time period to a second time period to update the first 3D model based on comparisons between a second 3D model and first depth measurements of the first region. The computing system may detect whether the first region changed after the second time period based on comparisons between the second 3D model and the first depth measurements of the first region. The computing system may in response to detecting a change in the first region, update the first 3D model of the first region.

Adaptive model updates for dynamic and static scenes

In one embodiment, a method includes receiving sensor data of a scene captured using one or more sensors, generating (1) a number of virtual surfaces representing a number of detected planar surfaces in the scene and (2) a point cloud representing detected features of objects in the scene based on the sensor data, assigning each point in the point cloud to one or more of the number of virtual surfaces, generating occupancy volumes for each of the number of virtual surfaces based on the points assigned to the virtual surface, generating a datastore including the number of virtual surfaces, the occupancy volumes of each of the number of virtual surfaces, and a spatial relationship between the number of virtual surfaces, receiving a query, and sending a response to the query, the response including an identified subset of the plurality of virtual surfaces in the datastore that satisfy the query.

Environment model with surfaces and per-surface volumes

In one embodiment, a method includes generating an outputted sequence of warped images from a captured sequence of images. Using this captured sequence of images, a computing system may determine one or more three-dimensional locations of object features and a corresponding camera position for each image in the captured sequence of images. Utilizing the camera positions for each image, the computing system may determine a view path representing the perspective of a virtual camera. The computing system may identify one or more virtual camera positions for the virtual camera located on the view path, and subsequently warp one or more images from the sequence of captured images to represent the perspective of the virtual camera at each of the respective virtual camera positions. This results in a sequence of warped images that may be outputted for viewing and interaction on a client device.

Outputting warped images from captured video data

In one embodiment, a computing system may update a first 3D model of a region of an environment based on comparisons between the first 3D model and first depth measurements of the region generated during a first time period. The computing system may determine that the region is static by comparing the first 3D model to second depth measurements of the region generated during a second time period. The computing system may in response to determining that the region is static, detect whether the region changed after the second time period based on comparisons between a second 3D model of the region and third depth measurements of the region generated after the second time period, the second 3D model having a lower resolution than the first 3D model. The computing system may in response to detecting a change in the region, update the first 3D model of the region.

In one embodiment, a method includes accessing a plurality of points, wherein each point (1) corresponds to a spatial location associated with an observed feature of a physical environment and (2) is associated with a patch representing the observed feature, determining a density associated with each of the plurality of points based on the spatial locations of the plurality of points, scaling the patch associated with each of the plurality of points based on the density associated with the point, and reconstructing a scene of the physical environment based on at least the scaled patches.

Growing community of inventors

Zurich, Switzerland

Fabian Langguth