Meta Platforms Technologies, LLC

Ml Netherlands C.v.

Magic Leap, Inc.

Facebook Technologies, LLC

Meta Platforms, Inc.

Mn Netherlands C.v.

Manuel Werlberger

Alexandru-Eugen Ichim

Gaurav Chaurasia

Alexander Sorkine Hornung

Alexander Ilic

Matthew James Alderman

Jan Oberländer

Gioacchino Noris

James Allan Booth

Benedikt Koeppel

Alex Locher

Boyang Zhang

Yuichi Taguchi

Yifu Zhang

A method of interacting with an artificial-reality (AR) content at an AR headset that includes cameras and displays is described. The method includes, while the AR headset has a first position within a physical environment that includes an object, if a distance of the object is within a threshold collision distance from the AR headset, presenting a meshed representation of the object. The meshed representation is displayed at first respective locations on the displays such that the meshed representation is viewable within the artificial reality. After the AR headset moves to a second position with a different distance, and if the different distance is within the threshold collision distance from the AR headset, moving the meshed representation of the object to second respective locations within the artificial reality. The second respective locations correspond to the position of the object in the physical environment.

Presenting meshed representations of physical objects within defined boundaries for interacting with artificial-reality content, and systems and methods of use thereof

In particular embodiments, a computing system may divide at least a portion of a physical space surrounding a user into a plurality of three-dimensional (3D) regions, wherein each of the 3D regions is associated with an area of a plurality of areas in a plane. The system may generate estimated locations of features of objects in the portion of the physical space. Based on the estimated locations, the system may determine an occupancy state of each of the plurality of 3D regions. Then based on the occupancy states of the plurality of 3D regions, the system may determine that one or more of the plurality of areas have respective airspaces that are likely unoccupied by objects.

Systems, methods, and media for detecting object-free space

A mixed reality (MR) simulation system includes a console and a head mounted device (HMD). The MR system captures stereoscopic images from a real-world environment using outward-facing stereoscopic cameras mounted to the HMD. The MR system preprocesses the stereoscopic images to maximize contrast and then extracts a set of features from those images, including edges or corners, among others. For each feature, the MR system generates one or more two-dimensional (2D) polylines. Then, the MR system triangulates between 2D polylines found in right side images and corresponding 2D polylines found in left side images to generate a set of 3D polylines. The MR system interpolates between 3D vertices included in the 3D polylines or extrapolates additional 3D vertices, thereby generating a geometric reconstruction of the real-world environment. The MR system may map textures derived from the real-world environment onto the geometric representation faster than the geometric reconstruction is updated.

Reconstruction of essential visual cues in mixed reality applications

A method of forming a visual representation of an object from a plurality of image frames acquired using a portable electronic device, the method comprising the steps of determining at least one parameter of motion of the portable electronic device; determining at least one capture condition for at least one first image frame of the plurality of image frames; computing, based on the at least one parameter of motion and the at least one capture condition, an indication of blur in the image frame; based on the indication of blur, conditionally taking a corrective action.

Adaptive camera control for reducing motion blur during real-time image capture

A smartphone may be freely moved in three dimensions as it captures a stream of images of an object. Multiple image frames may be captured in different orientations and distances from the object and combined into a composite image representing an image of the object. The image frames may be formed into the composite image based on representing features of each image frame as a set of points in a three dimensional point cloud. Inconsistencies between the image frames may be adjusted when projecting respective points in the point cloud into the composite image. Quality of the image frames may be improved by processing the image frames to correct errors. Further, operating conditions may be selected, automatically or based on instructions provided to a user, to reduce motion blur. Techniques, including relocalization such that, allow for user-selected regions of the composite image to be changed.

Dynamic updating of a composite image

Growing community of inventors

Zurich, Switzerland

Manuel Werlberger