Apparatus and method for synthesizing 2d image and media using inverse rendering with 3d spatially varying lighting information estimation

Abstract

The present disclosure relates to an apparatus for synthesizing a 2D image and media using a reverse rendering including 3D spatially varying lighting information estimation, and the apparatus includes a target view analysis unit configured to estimate a normal map and direct lighting using a plurality of 2D images, a material estimation unit configured to reflect the normal map and direct lighting estimated from the target view analysis unit and estimate material information, a 3D lighting estimation unit configured to reflect the direct lighting estimated from the target view analysis unit and the material information estimated from the material estimation unit and estimate 3D spatially varying lighting information, and an image generation unit configured to reflect the estimated shapes, materials, and 3D spatially varying lighting information, synthesize the 2D images with 3D objects, and generate a synthesized image.