Neural network layer folding

Abstract

The present disclosure describes neural network reduction techniques for decreasing the number of neurons or layers in a neural network. Embodiments of the method, apparatus, non-transitory computer readable medium, and system are configured to receive a trained neural network and replace certain non-linear activation units with an identity function. Next, linear blocks may then be folded to form a single block in places where the non-linear activation units were replaced by an identity function. Such techniques may reduce the number of layers in the neural network, which may optimize power and computation efficiency of the neural network architecture (e.g., without unduly influencing the accuracy of the network model).