Systems and methods for unsupervised learning of geometry from images using depth-normal consistency

Abstract

Presented are systems and methods for 3D reconstruction from videos using an unsupervised learning framework for depth and normal estimation via edge-aware depth-normal consistency. In embodiments, this is accomplished by using a surface normal representation. Depths may be reconstructed in a single image by watching unlabeled videos. Depth-normal regularization constrains estimated depths to be compatible with predicted normals, thereby, yielding geometry-consistency and improving evaluation performance and training speed. In embodiments, a consistency term is solved by constructing depth-to-normal layer and normal-to-depth layers within a deep convolutional network (DCN). In embodiments, the depth-to-normal layer uses estimated depths to compute normal directions based on neighboring pixels. Given the estimated normals, the normal-to-depth layer may then output a regularized depth map. Both layers may be computed with awareness of edges within the image. Finally, to train the network, the photometric error and gradient smoothness for both depth and normal predictions may be applied.