Multiresolution truncated neural radiance fields

Abstract

A system and method for receiving colour images, depth images and viewpoint information; dividing 3D space occupied by real-world environment into 3D grid(s) of voxels; create 3D data structure(s) comprising nodes, each node representing corresponding voxel; dividing colour image and depth image into colour tiles and depth tiles, respectively; mapping colour tile to voxel(s) whose colour information is captured in colour tile; storing, in node representing voxel(s), viewpoint information indicative of viewpoint from which colour and depth images are captured, along with any of: colour tile that captures colour information of voxel(s) and corresponding depth tile that captured depth information, or reference information indicative of unique identification of colour tile and corresponding depth tile; and utilising 3D data structure(s) for training neural network(s), wherein input of neural network(s) comprises 3D position of point and output of neural network(s) comprises colour and opacity of point.