Facebook Technologies, LLC

Meta Platforms Technologies, LLC

Meta Platforms, Inc.

Alex Locher

Alexander Sorkine Hornung

Gioacchino Noris

Alexandru-Eugen Ichim

Gaurav Chaurasia

Manuel Werlberger

James Allan Booth

Naveen Makineni

John Enders Robertson

Anthony Aslan Tenggoro

Oskar Linde

A mixed reality (MR) simulation system includes a console and a head mounted device (HMD). The MR system captures stereoscopic images from a real-world environment using outward-facing stereoscopic cameras mounted to the HMD. The MR system preprocesses the stereoscopic images to maximize contrast and then extracts a set of features from those images, including edges or corners, among others. For each feature, the MR system generates one or more two-dimensional (2D) polylines. Then, the MR system triangulates between 2D polylines found in right side images and corresponding 2D polylines found in left side images to generate a set of 3D polylines. The MR system interpolates between 3D vertices included in the 3D polylines or extrapolates additional 3D vertices, thereby generating a geometric reconstruction of the real-world environment. The MR system may map textures derived from the real-world environment onto the geometric representation faster than the geometric reconstruction is updated.

Reconstruction of essential visual cues in mixed reality applications

In one embodiment, a computing system may receive sensor data from an image sensor having a pixel array including color pixel sensors and panchromatic pixel sensors in a first pattern. Each of the color pixel sensors is associated with one of several color channels. The computing system may generate, based on the sensor data, a filtered monochrome image including monochrome values corresponding to the pixel array of the image sensor. The computing system may generate a filtered color image having a second pattern of color channels. A first pixel of a particular color channel at a first pixel location in the filtered color image is determined based on the monochrome value corresponding to the first pixel location in the filtered monochrome image, the sensor data measured by a color pixel sensor at a second pixel location, and the monochrome value at the second pixel location in the filtered monochrome image.

Sparse RGB cameras for image capture

Growing community of inventors

Oberrohrdorf, Switzerland

Alex Locher