Systems and methods for training a depth estimator to be scale-aware using weak sonar supervision

Abstract

Systems and methods described herein relate to training a depth estimator to be scale-aware using weak sonar supervision. In one embodiment, a depth and scale training system trains a self-supervised monocular depth estimation (MDE) network by minimizing a photometric reprojection error between an original target image of a scene and a viewpoint-synthesized image. The depth and scale training system also supervises weakly the training of the MDE network by minimizing a loss computed between a depth image generated by the MDE network and a set of sparse sonar data points collected from the scene to teach the MDE network to estimate metric scale in connection with controlling a robot.