Methods and apparatus for augmenting dense depth maps using sparse data

Abstract

Systems, apparatus, and methods for augmenting dense depth maps using sparse data. Various embodiments combine single-image depth estimation (SIDE) techniques with structure-from-motion techniques for improved depth accuracy. In some examples, a machine learning (ML) model is used to generate a dense depth map based on one or more frames/images of a video. Structure-from-motion (SfM) analysis is performed on the video to determine depth information from camera movement in the video. The structure-from-motion techniques may generate more accurate data than the ML model, however, the data from the ML model may be denser compared with the SFM depth data. The dense ML model depth map may be augmented by the SFM depth data. Augmentation may include fitting the relative depths determined by the ML model depth map to the absolute depth in the SFM data resulting in more accurate dense depth information.