Intel Corporation

Apple Inc.

Samer Samir Barakat

Mohamed Selim Ben Himane

Haowei Liu

Vincent Chapdelaine-Couture

Bertrand Nepveu

Daniel Joseph Mirota

Duc Q Pham

In one implementation, a method of performing perspective correction is performed at a head-mounted device including at a head-mounted device including one or more processors, non-transitory memory, an image sensor, and a display. The method includes capturing, using the image sensor, an image of a scene. The method includes obtaining a plurality of depths respectively associated with a plurality of pixels of the image of the scene. The method includes generating a smooth depth map of the image of the scene based on the plurality of depths, wherein a difference between any two adjacent elements of the depth map is below a threshold. The method includes transforming, using the one or more processors, the image of the scene based on the smooth depth map and a difference between a perspective of the image sensor and a perspective of a user. The method includes displaying, on the display, the transformed image.

Method and device for perspective correction using a smooth depth map

In one implementation, a method of performing perspective correction is performed at a head-mounted device including one or more processors, non-transitory memory, an image sensor, and a display. The method includes capturing, using the image sensor, a plurality of images of a scene from a respective plurality of perspectives. The method includes capturing, using the image sensor, a current image of the scene from a current perspective. The method includes obtaining a depth map of the current image of the scene. The method include transforming, using the one or more processors, the current image of the scene based on the depth map, a difference between the current perspective of the image sensor and a current perspective of a user, and at least one of the plurality of images of the scene from the respective plurality of perspectives. The method includes displaying, on the display, the transformed image.

Method and device for perspective correction using one or more keyframes

Methods, apparatuses and systems may provide for conducting a quality assessment of a depth localization mode, a color localization mode and an inertia localization mode, and selecting one of the depth localization mode, the color localization mode or the inertia localization mode as an active localization mode based on the quality assessment. Additionally, a pose of a camera may be determined relative to a three-dimensional (3D) environment in accordance with the active localization mode.

Multi-modal real-time camera localization and environment mapping

Techniques are disclosed for drift correction for camera tracking. In some cases, the techniques include iterative optimization of the camera poses at selected keyframes captured along the camera trajectory to reduce pose errors. Such iterative optimization may include performing an alignment process using the point cloud of a given keyframe and a point cloud made from one or more overlapping keyframes, such as via an iterative closest point (ICP)-based expectation maximization. The keyframes may then be fused to reconstruct a more accurate model of the scene, discarding the existing model. The new model can then be used for tracking and meshing. In some instances, keyframes may be selected such that the overlap with other keyframes includes enough shape features to allow for alignment with the keyframe depth point cloud. In some cases, the techniques can be performed in a particular order giving precedence to keyframes having smaller pose errors.

Drift correction for camera tracking

A system for place recognition is described herein. The system for place recognition comprises a plurality of sensors, a memory, and a processor. The memory is to store instructions and is communicatively coupled to the plurality of sensors. The processor is communicatively coupled to the plurality of sensors and the memory. When the processor is to execute the instructions, the processor is to detect features in a current frame and extract descriptors of the features of the current frame. The processor is also to generate a vocabulary tree using the descriptors and determine candidate key frames based on the vocabulary tree and detected features. The processor also is to perform place recognition via a first stage matching and a second stage matching.

Growing community of inventors

Santa Clara, CA, United States of America

Samer Samir Barakat