Synthetic depth image generation from cad data using generative adversarial neural networks for enhancement

Abstract

A system and method for generating realistic depth images by enhancing simulated images rendered from a 3D model, include a rendering engine configured to render noiseless 2.5D images by rendering various poses with respect to a target 3D CAD model, a noise transfer engine configured to apply realistic noise to the noiseless 2.5D images, and a background transfer engine configured to add pseudo-realistic scenedependent backgrounds to the noiseless 2.5D images. The noise transfer engine is configured to learn noise transfer based on a mapping, by a first generative adversarial network (GAN), of the noiseless 2.5D images to real 2.5D scans generated by a targeted sensor. The background transfer engine is configured to learn background generation based on a processing, by a second GAN, of output data of the first GAN as input data and corresponding real 2.5D scans as target data.