Facebook, Inc.

Facebook Technologies, LLC

Meta Platforms Technologies, LLC

David James Geraghty

Richard S Szeliski

Daniel Scharstein

An artificial reality system is configured to more accurately and efficiently construct a 3D virtual representation of a real-world environment from a set of 2D images. The system identifies points and/or lines within the images that define a plane along an orientation and then performs a planar sweep along a perpendicular path to identify surfaces in which the plane intersects with multiple points. Points that appear to be in the same plane are 'collapsed' into a cohesive plane to conserve processing power by estimating and/or storing parameters for the cohesive plane, rather than all of the individual 3D points. In this way, the system also 'averages out' random variation in the planar surface that would otherwise result from random noise in the estimation of the individual 3D points. The system may then generate a 3D map from a constrained alignment of all of the identified planes.

Artificial reality system with 3D environment reconstruction using planar constraints

In one embodiment, a method includes accessing an image of a physical world scene, detecting a number of straight-line segments in the accessed image, identifying a first vanishing point and a second vanishing point in the image, where each vanishing point corresponds to a sub-set of the number of straight-line segments, and where the first vanishing point and the second vanishing point are orthogonal, identifying a planar region in the accessed image represented by one or more straight-line segments associated with the first vanishing point and one or more straight-line segments associated with the second vanishing point, transforming a virtual object associated with the planar region based on one or more properties associated with the planar region, and displaying the transformed virtual object over the image.

Growing community of inventors

Seattle, WA, United States of America

David James Geraghty