Virtual viewpoint image synthesis device and method using quasi-uniform spherical coordinate grid

Abstract

An image synthesis device includes a memory storing a virtual viewpoint image synthesis program, and a processor configured to execute the program. The program receives images of multiple viewpoints obtained by filming around a user, position information on the user, and information on a direction in which scene is viewed, estimates color and density by inputting the position information of the user and the direction in which the object is viewed to a neural radiance field model constructed by using the images of the multiple viewpoints, and synthesizes images of a virtual viewpoint by performing volume rendering by using estimated color and density, the neural network radiance field model is constructed based on a quasi-uniform spherical coordinate system, and the quasi-uniform spherical coordinate system includes a sum of a Yin grid and a Yang grid and has a grid structure that increases exponentially in a radial direction.