Solving the distal reward problem through linkage of stdp and dopamine signaling

Abstract

In Pavlovian and instrumental conditioning, rewards typically come seconds after reward-triggering actions, creating an explanatory conundrum known as the distal reward problem or the credit assignment problem. How does the brain know what firing patterns of what neurons are responsible for the reward if (1) the firing patterns are no longer there when the reward arrives and (2) most neurons and synapses are active during the waiting period to the reward? A model network and computer simulation of cortical spiking neurons with spike-timing-dependent plasticity (STDP) modulated by dopamine (DA) is disclosed to answer this question. STDP is triggered by nearly-coincident firing patterns of a presynaptic neuron and a postsynaptic neuron on a millisecond time scale, with slow kinetics of subsequent synaptic plasticity being sensitive to changes in the extracellular dopamine DA concentration during the critical period of a few seconds after the nearly-coincident firing patterns.