Radiance fields for three-dimensional reconstruction and novel view synthesis in large-scale environments

Abstract

Systems and methods for view synthesis and three-dimensional reconstruction can learn an environment by utilizing a plurality of images of an environment and depth data. The use of depth data can be helpful when the quantity of images and different angles may be limited. For example, large outdoor environments can be difficult to learn due to the size, the varying image exposures, and the limited variance in view direction changes. The systems and methods can leverage a plurality of panoramic images and corresponding lidar data to accurately learn a large outdoor environment to then generate view synthesis outputs and three-dimensional reconstruction outputs. Training may include the use of an exposure correction network to address lighting exposure differences between training images.