Building a state space for song learning
Building a state space for song learning
复制标题
DOI：
10.1016/j.conb.2017.12.001
复制
发表时间：
2018-04-01
影响因子：
5.7
通讯作者：
Fee, Michale Sean
中科院分区：
文献类型：
--
作者：
Mackevicius, Emily Lambert;Fee, Michale Sean
鸣禽系统揭示了大脑如何产生精确定时的行为序列，以及大脑如何实施强化学习（RL）。强化学习是一种强大的策略，用于学习在每个状态下产生什么动作，但需要任务中涉及的状态的唯一表示。人们认为，Songbird RL 电路使用歌曲音节中每个时刻的表示来进行操作，这与前运动皮层核 HVC 中神经元的稀疏顺序爆发一致。然而，这种稀疏的序列并不存在于非常幼小的鸟类中，它们会唱出长度随机的高度可变的音节。在这里，我们根据阐明 HVC 和听觉皮层区域之间联系的新数据，回顾并扩展了鸣禽大脑如何构建潜在序列来支持 RL 的模型。我们假设学习通过四个不同的可塑性过程发生：1）听觉区域“导师记忆”序列的形成； 2) 形成适当定时的潜在 HVC 序列，通过听觉区域自发重放导师歌曲的输入进行播种； 3) 在自发重放期间，加强从 HVC 到“导师记忆”序列中相应时序的听觉神经元的连接，调整听觉和运动表征以用于后续歌曲评估； 4) 通过详细描述的歌曲 RL 电路，加强从前运动神经元到产生所需声音的运动输出神经元的连接。
The songbird system has shed light on how the brain produces precisely timed behavioral sequences, and how the brain implements reinforcement learning (RL). RL is a powerful strategy for learning what action to produce in each state, but requires a unique representation of the states involved in the task. Songbird RL circuitry is thought to operate using a representation of each moment within song syllables, consistent with the sparse sequential bursting of neurons in premotor cortical nucleus HVC. However, such sparse sequences are not present in very young birds, which sing highly variable syllables of random lengths. Here, we review and expand upon a model for how the songbird brain could construct latent sequences to support RL, in light of new data elucidating connections between HVC and auditory cortical areas. We hypothesize that learning occurs via four distinct plasticity processes: 1) formation of 'tutor memory' sequences in auditory areas; 2) formation of appropriately-timed latent HVC sequences, seeded by inputs from auditory areas spontaneously replaying the tutor song; 3) strengthening, during spontaneous replay, of connections from HVC to auditory neurons of corresponding timing in the 'tutor memory' sequence, aligning auditory and motor representations for subsequent song evaluation; and 4) strengthening of connections from premotor neurons to motor output neurons that produce the desired sounds, via well-described song RL circuitry.