System and method for compressing convolutional neural networks

Abstract

An apparatus is provided to compress CNN models using a combination of filter pruning and tensor decomposition. For example, the apparatus accesses a trained CNN that includes convolutional tensors. The apparatus prunes the filters of a convolutional tensor to generate a sparse tensor. Further, the apparatus decomposes the sparse tensor to generate a low-rank approximation of the sparce tensor. The low-rank approximation of the sparse tensor includes a core tensor and principal tensors. The apparatus generates a convolutional flow that includes the core tensor and convolutional operations generated based on the principal tensors. The apparatus may replace some or all the convolutional tensors in the trained CNN with the corresponding convolutional flows. The apparatus may fine-tune the updated CNN by re-training the updated CNN. The number of epochs for re-training the updated CNN may be smaller than the number of epochs for training the CNN.