Camera parameter preconditioning for neural radiance fields

Abstract

Systems and methods for training a machine-learned model are disclosed herein. The method can include obtaining, by a processor, a plurality of images, each image having a set of parameter values comprising values for a plurality of camera parameters and determining a covariance matrix for the plurality of camera parameters with respect to a plurality of projected points generated via evaluation of a projection function. The method can also include performing a whitening algorithm to identify a preconditioning matrix that, when applied to the sets of parameter values, results in the covariance matrix being approximately equal to an identity matrix and performing an optimization algorithm on the plurality of sets of parameter values, Performing the optimization algorithm can include applying an inverse of the preconditioning matrix to the plurality of sets of parameters in a forward prediction pass and applying the preconditioning matrix in a backward gradient pass.