Robert Bosch

Zhixin Yan

Liu Ren

Mao Ye

Benzun Pious Wisely Babu

Ye Duan

Yuyan Li

Soohwan Kim

Liang Mi

A system and method is disclosed having an end-to-end two-stage depth estimation deep learning framework that takes one spherical color image and estimate dense spherical depth maps. The contemplated framework may include a view synthesis (stage 1) and a multi-view stereo matching (stage 2). The combination of the two-stage process may provide the advantage of the geometric constraints from stereo matching to improve depth map quality, without the need of additional input data. It is also contemplated that a spherical warping layer may be used to integrate multiple spherical features volumes to one cost volume with uniformly sampled inverse depth for the multi-view spherical stereo matching stage. The two-stage spherical depth estimation system and method may be used in various applications including virtual reality, autonomous driving and robotics.

Two-stage depth estimation machine learning algorithm and spherical warping layer for equi-rectangular projection stereo matching

Two-stage depth estimation machine learning algorithm and spherical warping layer for EQUI-rectangular projection stereo matching

Depth perception has become of increased interest in the image community due to the increasing usage of deep neural networks for the generation of dense depth maps. The applications of depth perception estimation, however, may still be limited due to the needs of a large amount of dense ground-truth depth for training. It is contemplated that a self-supervised control strategy may be developed for estimating depth maps using color images and data provided by a sensor system (e.g., sparse LiDAR data). Such a self-supervised control strategy may leverage superpixels (i.e., group of pixels that share common characteristics, for instance, pixel intensity) as local planar regions to regularize surface normal derivatives from estimated depth together with the photometric loss. The control strategy may be operable to produce a dense depth map that does not require a dense ground-truth supervision.

Self-supervised depth estimation method and system

A visual SLAM system comprises a plurality of keyframes including a keyframe, a current keyframe, and a previous keyframe, a dual dense visual odometry configured to provide a pairwise transformation estimate between two of the plurality of keyframes, a frame generator configured to create keyframe graph, a loop constraint evaluator adds a constraint to the receiving keyframe graph, and a graph optimizer configured to produce a map with trajectory.

RGB-D camera based tracking system and method thereof

A method for motion estimation in an augmented reality (AR) system includes receiving inertial sensor data and image data during movement of the AR system generating a probability map based on the inertial sensor data and the image data, the probability map corresponding to one frame in the image data and including probability values indicating that each pixel in the one frame is in an inertial coordinate frame or a local coordinate frame with a convolutional neural network encoder/decoder, identifying visual observations of at least one landmark in the local coordinate frame based on the image data and the probability map, and generating an estimate of secondary motion in the local coordinate frame based on a first prior state in a hidden Markov model (HMM) corresponding to the local coordinate frame and the visual observations of the at least one landmark in the local coordinate frame.

Methods and systems for exploiting per-pixel motion conflicts to extract primary and secondary motions in augmented reality systems

A novel map representation called Probabilistic Surfel Map (PSM) for dense visual SLAM is disclosed. The PSM maintains a globally consistent map with both photometric and geometric uncertainties encoded in order to address inconsistency and sensor noise. A key aspect of the visual SLAM method disclosed herein is the proper modeling and updating of the photometric and geometric uncertainties encoded in the PSM. Strategies for applying the PSM for improving both the front-end pose estimation and the back-end optimization are disclosed. Moreover, the PSM enables generation of a high quality dense point cloud with high accuracy.

Growing community of inventors

Sunnyvale, CA, United States of America

Zhixin Yan