Selective neural network pruning by masking filters using scaling factors

Abstract

A method and system for pruning a neural network (NN) block of a neural network during training, wherein the NN block comprises: a convolution operation configured to convolve an input feature map with a plurality of filters, each filter including a plurality of weights, to generate a plurality of filter outputs each corresponding to a respective filter; an activation operation configured to generate, for each of the filter outputs, a respective non-linearized output; a scaling operation configured to scale the non-linearized output generated in respect of each filter by multiplying the non-linearized output with a mask function and a respective scaling factor that corresponds to the filter. During training: for each scaling factor corresponding to a filter, learning the scaling factor by minimizing loss of a loss function including a first regularization function with respect to the scaling factor; and if a value of the scaling factor satisfies a predetermined criterion, selectively pruning the filter corresponding to the scaling factor by masking the filter from the convolution operation.