Neural network robustness via binary activation

Abstract

A method of increasing neural network robustness. The method comprises defining an artificial neural network comprising a number of bounded ramp activation functions. The network is trained iteratively in a layer-by-layer fashion. Each iteration increases the slope of the activation functions toward a discrete threshold activation and stops when the activation functions converge to the threshold activation and the network exhibits spiking behavior. Alternatively, weight agnostic neural networks are created, wherein nodes in the networks comprise fixed shared weights. A subset of networks is identified that comprise activation functions compatible with neuromorphic hardware and are tested with a specified number of shared weight values. A score is generated for each combination of network and weight value according to performance and mapping to neuromorphic hardware, and the networks are ranked. The networks are then combined according to ranking to create a new network that exhibits spiking behavior.