Solving the distal reward problem through linkage of STDP and dopamine signaling
Solving the distal reward problem through linkage of STDP and dopamine signaling
复制标题
DOI：
10.1093/cercor/bhl152
复制
发表时间：
2007-10-01
期刊：
影响因子：
3.7
通讯作者：
Izhikevich, Eugene M.
中科院分区：
文献类型：
--
作者：
Izhikevich, Eugene M.
在巴甫洛夫条件反射和工具性条件反射中，奖赏通常是在奖赏触发动作的几秒钟后到来的，这就产生了一个被称为“远端奖赏问题”的解释性难题：如果1)奖赏到达时这些模式不再存在，2)所有神经元和突触在奖赏的等待期内都是活跃的，大脑如何知道负责奖赏的神经元的什么放电模式？在这里，我们展示了这个难题是如何通过皮层棘波神经元的模型网络来解决的，该模型网络具有多巴胺(DA)调制的峰时依赖可塑性(STDP)。尽管STDP是由毫秒尺度上几乎一致的放电模式触发的，但随后突触可塑性的缓慢动力学对关键时期细胞外DA浓度的变化很敏感。在奖励的等待期内的随机触发不会影响STDP，因此使网络对正在进行的活动不敏感这是我们的方法与以前的理论研究不同的关键特征，后者隐含地假设网络在等待期间是安静的，或者模式被保留到奖励到达。这项研究强调了精确的放电模式在脑动力学中的重要性，并提出了细胞外DA形式的全球扩散强化信号如何在正确的时间选择性地影响右侧突触。
In Pavlovian and instrumental conditioning, reward typically comes seconds after reward-triggering actions, creating an explanatory conundrum known as "distal reward problem": How does the brain know what firing patterns of what neurons are responsible for the reward if 1) the patterns are no longer there when the reward arrives and 2) all neurons and synapses are active during the waiting period to the reward? Here, we show how the conundrum is resolved by a model network of cortical spiking neurons with spike-timing-dependent plasticity (STDP) modulated by dopamine (DA). Although STDP is triggered by nearly coincident firing patterns on a millisecond timescale, slow kinetics of subsequent synaptic plasticity is sensitive to changes in the extracellular DA concentration during the critical period of a few seconds. Random firings during the waiting period to the reward do not affect STDP and hence make the network insensitive to the ongoing activity the key feature that distinguishes our approach from previous theoretical studies, which implicitly assume that the network be quiet during the waiting period or that the patterns be preserved until the reward arrives. This study emphasizes the importance of precise firing patterns in brain dynamics and suggests how a global diffusive reinforcement signal in the form of extracellular DA can selectively influence the right synapses at the right time.