Xilinx, Inc.

Nicholas Fraser

Michaela Blott

Kornelis Antonius Vissers

Kristof Denolf

Yaman Umuroglu

Julian Faraone

Giulio Gambardella

Ussama Zahid

An example method of training a neural network includes defining hardware building blocks (HBBs), neuron equivalents (NEQs), and conversion procedures from NEQs to HBBs; defining the neural network using the NEQs in a machine learning framework; training the neural network on a training platform; and converting the neural network as trained into a netlist of HBBs using the conversion procedures to convert the NEQs in the neural network to the HBBs of the netlist.

Learning neural networks of programmable device blocks directly with backpropagation

Examples herein propose operating redundant ML models which have been trained using a boosting technique that considers hardware faults. The embodiments herein describe performing an evaluation process where the performance of a first ML model is measured in the presence of a hardware fault. The errors introduced by the hardware fault can then be used to train a second ML model. In one embodiment, a second evaluation process is performed where the combined performance of both the first and second trained ML models is measured in the presence of a hardware fault. The resulting errors can then be used when training a third ML model. In this manner, the three trained ML models are trained to be error aware. As a result, during operation, if a hardware fault occurs, the three ML models have better performance relative to three ML models that where not trained to be error aware.

Error aware module redundancy for machine learning

A neural network system includes an input layer, one or more hidden layers, and an output layer. A first layer circuit implements a first layer of the one or more hidden layers. The first layer includes a first weight space including one or more subgroups. A forward path circuit of the first layer circuit includes a multiply and accumulate circuit to receive an input from a layer preceding the first layer; and provide a first subgroup weighted sum using the input and a first plurality weights associated with a first subgroup. A scaling coefficient circuit provides a first scaling coefficient associated with the first subgroup, and applies the first scaling coefficient to the first subgroup weighted sum to generate a first subgroup scaled weighted sum. An activation circuit generates an activation based on the first subgroup scaled weighted sum and provide the activation to a layer following the first layer.

System and method for implementing neural networks in integrated circuits

A neural network system includes an input layer, one or more hidden layers, and an output layer. The input layer receives a training set including a sequence of batches and provides to its following layer output activations associated with the sequence of batches respectively. A first hidden layer receives, from its preceding layer, a first input activation associated with a first batch, receive a first input gradient associated with a second batch preceding the first batch, and provide, to its following layer a first output activation associated with the first batch based on the first input activation and first input gradient. The first and second batches have a delay factor associated with at least two batches. The output layer receives, from its preceding layer, a second input activation, and provide, to its preceding layer, a first output gradient based on the second input activation and the first training set.

Growing community of inventors

Dublin, Ireland

Nicholas Fraser