Dynamic pruning of neurons on-the-fly to accelerate neural network inferences

Abstract

Systems, apparatuses and methods may provide for technology that aggregates contextual information from a first network layer in a neural network having a second network layer coupled to an output of the first network layer, wherein the context information is to be aggregated in real-time and after a training of the neural network, and wherein the context information is to include channel values. Additionally, the technology may conduct an importance classification of the aggregated context information and selectively exclude one or more channels in the first network layer from consideration by the second network layer based on the importance classification.