Motion-aware neural radiance field network training

Abstract

A method includes obtaining multiple training image frames of a scene, where the training image frames are captured at multiple viewpoints and multiple viewing angles relative to the scene. The method also includes generating multiple initial motion maps using the training image frames and identifying three-dimensional (3D) feature points associated with the scene using the training image frames. The method further includes generating tuned motion masks using the initial motion maps and projections of the 3D feature points onto the initial motion maps. In addition, the method includes training a machine learning model using the training image frames and the tuned motion masks, where the machine learning model is trained to generate 3D information about the scene from viewpoints and viewing angles not captured in the training image frames.