Self extrinsic self-calibration via geometrically consistent self-supervised depth and ego-motion learning

Abstract

Systems and methods described herein relate to self-supervised scale-aware learning of camera extrinsic parameters. One embodiment processes instantaneous velocity between a target image and a context image captured by a first camera; jointly training a depth network and pose network based on scaling by the instantaneous velocity; produce depth map using the depth network; produce ego-motion of the first camera using the pose network; generate synthesized image from the target image using a reprojection operation based on the depth map, the ego-motion, the context image and camera intrinsics; determine photometric loss by comparing the synthesized image to the target image; generate photometric consistency constraint using a gradient from the photometric loss; determine pose consistency constraint between the first camera and a second camera; and optimize the photometric consistency constraint, the pose consistency constraint, the depth network and the pose network to generate estimated extrinsic parameters.