Reinforcement learning with modulated spike timing-dependent synaptic plasticity
Reinforcement learning with modulated spike timing-dependent synaptic plasticity
复制标题
DOI：
10.1152/jn.00364.2007
复制
发表时间：
2007-12-01
影响因子：
2.5
通讯作者：
Fairhall, Adrienne L.
中科院分区：
文献类型：
--
作者：
Farries, Michael A.;Fairhall, Adrienne L.
Spike - time -dependent synaptic plasticity （STDP）被认为是连接突触前和突触后活动模式与突触强度变化的首选框架。尽管突触可塑性被广泛认为是学习的主要组成部分，但目前尚不清楚STDP本身如何作为通用学习的机制。另一方面，强化学习的算法适用于各种各样的问题，但缺乏实验建立的神经实现。在这里，我们将这些范式结合在一个新的模型中，其中STDP的修改版本实现了强化学习。我们分阶段建立这个模型，确定使其工作所需的最小条件集。在两层前馈网络中使用性能调制的STDP修改，我们可以训练输出神经元来产生任意选择的尖峰序列或群体响应。此外，给定的网络可以学习对几种不同输入模式的不同响应。我们还详细描述了该模型如何在生物学上实现。因此，我们的模型提供了一种新颖的、生物学上合理的强化学习实现，能够训练神经群体在突触输入和尖峰输出之间产生非常广泛的可能映射。
Spike timing-dependent synaptic plasticity (STDP) has emerged as the preferred framework linking patterns of pre- and postsynaptic activity to changes in synaptic strength. Although synaptic plasticity is widely believed to be a major component of learning, it is unclear how STDP itself could serve as a mechanism for general purpose learning. On the other hand, algorithms for reinforcement learning work on a wide variety of problems, but lack an experimentally established neural implementation. Here, we combine these paradigms in a novel model in which a modified version of STDP achieves reinforcement learning. We build this model in stages, identifying a minimal set of conditions needed to make it work. Using a performance-modulated modification of STDP in a two-layer feedforward network, we can train output neurons to generate arbitrarily selected spike trains or population responses. Furthermore, a given network can learn distinct responses to several different input patterns. We also describe in detail how this model might be implemented biologically. Thus our model offers a novel and biologically plausible implementation of reinforcement learning that is capable of training a neural population to produce a very wide range of possible mappings between synaptic input and spiking output.