System and method for parameter multiplexed gradient descent

Abstract

Embodiments of the present invention relate to systems and model-free methods for perturbing neural network hardware parameters and measure the neural network response that are implemented natively within the neural network hardware and without requiring a knowledge of the internal structure of the network. Embodiments of the present invention also relate to systems and methods for configuring neural network hardware such that the network automatically performs parameter multiplexed gradient descent, which include adding a time-varying perturbation to each hardware parameter base value to modulate the cost, broadcasting the modulated cost signal to all hardware parameters, and filtering out modulations so as to extract gradient information.