Training of differentiable renderer and neural network for query of 3d model database

Abstract

System and method for differentiable networks trainable to learn an optimized query of a 3D model database used for object recognition includes training a first differentiable network configured as a differentiable renderer by generating 2D images from 3D models of a first object of a dissimilar second object while optimizing rendering parameters for producing 2D images by gradient descent of a first triple loss function. Visual variation among the images is maximized. A second differentiable network configured as a convolutional neural network defined by a regression function is trained by generating searchable feature vectors of the 2D images. The feature vectors are determined using optimized neural network parameters determined by gradient descent of a second triple loss function to achieve high correlation to an input image of the first object and low correlation to images of the second object.