Decomposing and recomposing scenes using neural radiance fields

Abstract

The present invention sets forth a technique for performing scene decomposition. This technique includes determining, based on a plurality of two-dimensional (2D) representations of a three-dimensional (3D) scene, a different radiance field function for each of a plurality of neural radiance fields (NeRFs). The technique also includes generating a combined radiance field function based on the radiance field functions associated with the plurality of NeRFs. The technique further includes generating a color value for a given 3D location and viewing angle in the 3D scene. The technique further includes computing a decomposition loss based on the difference between the color value and a ground truth color value associated with the 3D location and viewing angle. The technique further includes modifying at least one of the plurality of NeRFs based on the decomposition loss such that each NeRF is associated with a different object in the 3D scene.