Heterogeneous convolutional neural network for multi-problem solving

Abstract

A heterogeneous convolutional neural network (HCNN) system includes a visual reception system generating an input image. A feature extraction layer (FEL) portion of convolutional neural networks includes multiple convolution, pooling and activation layers stacked together. The FEL includes multiple stacked layers, a first set of layers learning to represent data in a simple form including horizontal and vertical lines and blobs of colors. Following layers capture more complex shapes such as circles, rectangles, and triangles. Subsequent layers pick up complex feature combinations to form a representation including wheels, faces and grids. The FEL portion outputs data to each of: a first sub-network which performs a first task of object detection, classification, and localization for classes of objects in the input image to create a detected object table; and a second sub-network which performs a second task of defining a pixel level segmentation to create a segmentation data set.