Cascaded architecture for disparity and motion prediction with block matching and convolutional neural network (cnn)

Abstract

A CNN operates on the disparity or motion outputs of a block matching hardware module, such as a DMPAC module, to produce refined disparity or motion streams which improve operations in images having ambiguous regions. As the block matching hardware module provides most of the processing, the CNN can be small and thus able to operate in real time, in contrast to CNNs which are performing all of the processing. In one example, the CNN operation is performed only if the block hardware module output confidence level is below a predetermined amount. The CNN can have a number of different configurations and still be sufficiently small to operate in real time on conventional platforms.