Cerebellar circuits for reward-based learning

基于奖励的学习的小脑回路

• 批准号: 10710404
• 负责人: COURT A HULL
• 金额: $ 32.42万
• 依托单位: DUKE UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-28 至 2027-07-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10710404
• 关键词: Association Learning; Automobile Driving; Behavior; Behavioral; Behavioral Paradigm; Binding; Calcium; Calibration; Cerebellar Cortex; Cerebellum; Conditioned Stimulus; Cues; Data; Dendrites; Educational process of instructing; Electrodes; Evaluation; Fiber; Future; Generations; Image; Instruction; Learning; Life; Link; Measures; Mediating; Modeling; Monitor; Motor; Motor output; Movement; Mus; Outcome; Output; Pathway interactions; Play; Probability; Property; Psychological reinforcement; Purkinje Cells; Reflex action; Reporting; Response to stimulus physiology; Rewards; Signal Transduction; Site; Specificity; Stimulus; Synapses; System; Testing; Thinking; Training; Ventral Tegmental Area; Visualization; Walking; Work; Writing; dopaminergic neuron; experimental study; extracellular; fiber cell; granule cell; in vivo calcium imaging; learning strategy; mossy fiber; motor control; motor learning; optogenetics; postsynaptic; predictive modeling; response; reward anticipation; supervised learning; theories; transmission blocking; two-photon

ABSTRACT The cerebellum plays a key role in motor coordination and learning. Classic models posit that cerebellar learning is instructed by teaching signals from climbing fibers (CFs) that act according to the principles of supervised learning. While such models work well to describe CF activity and learning in some behaviors, they are not sufficient to explain CF activity in others. By developing an operant, reward-guided cerebellar-dependent task for the mouse, as well as a modified classical conditioning task, we used calcium imaging of CF input to Purkinje cell dendrites to demonstrate that CFs can be driven by reward-related task parameters. Our data suggested the possibility that CFs might engage in reinforcement learning to report predictions about expected rewards (reward prediction errors) in a similar manner as dopaminergic neurons of the ventral tegmental area (VTA). Importantly, however, our data also show significant differences from some predicitons of leading reinforcement learning models, and many other properties of cerebellar reward-based learning remain unclear. Thus, it remains largely unknown how the cerebellum operates in reward-based learning. Here will rigoursly test the hypothesis that CFs instruct cerbellar learning according to reinforcement learning rules: In the first aim, we will test whether CF activity obeys the many diverse requirements of reward prediction error signals, for example by scaling with both the probability and size of an expected reward. To do so, we will use two-photon calcium imaging to monitor CF input to Purkinje cell dendrites while manipulating reward contingencies during a classical conditioning paradigm. We will also determine the contribution of behavioral context, learning, and motor output to CF activity. In the second aim, we will test whether reward-predictive CF activity is generated by reward- responsive CF activity. This is a key property of reinforcement learning because it binds activity driven by an unconditioned simulus (US) to activity driven by a conditioned stimulus (CS). We will use classical blocking experiments and optogenetic manipulations to determine the neccessity and sufficiency of US-linked CF responses to generating CS-linked CF responses. Finally, in the third aim, we will determine whether CS-linked CF activity drives learned changes in behavior and cerebellar output. Thus, we will use a combination of optogenetics and extracellular electrophysioloigcal recordings test the function of reward-related CF activity. Together, these experiments will reveal the key principles that govern reward-based cerebellar learning, and how this learning alters cerebellar output and behavior.

摘要 小脑在运动协调和学习中起着关键作用。经典模型假设小脑学习 是由来自攀爬纤维(CF)的示教信号指示的，这些信号根据监督的原则行事 学习。虽然这样的模型很好地描述了某些行为中的CF活动和学习，但它们不是 足以解释其他人的CF活动。通过发展一种操作性的、奖赏引导的小脑依赖任务 对于小鼠，以及改进的经典条件反射任务，我们使用了对浦肯野输入的CF的钙成像 细胞树突实验证明，CFS可以由奖赏相关的任务参数驱动。我们的数据显示 CFS进行强化学习以报告对预期回报的预测的可能性 (奖赏预测错误)，其方式类似于腹侧被盖区(VTA)的多巴胺能神经元。 然而，重要的是，我们的数据也显示出与一些领先增援预测的显著差异 学习模型以及小脑奖赏学习的许多其他特性仍不清楚。因此，它 在很大程度上，人们仍然不知道小脑在基于奖励的学习中是如何运作的。在这里将严格测试 假设CFS根据强化学习规则指导小脑学习：在第一个目标中，我们将 测试CF活动是否符合奖励预测误差信号的多种不同要求，例如通过 根据预期回报的概率和规模进行调整。为此，我们将使用双光子钙 在经典实验期间，在操纵奖赏偶发事件的同时，监测CF对浦肯野细胞树突的输入的成像 条件反射范式。我们还将确定行为背景、学习和运动输出的贡献 对CF活性的影响。在第二个目标中，我们将测试奖励预测的CF活动是否由奖励产生- 响应的CF活动。这是强化学习的一个关键属性，因为它绑定了由 由条件刺激(CS)驱动的活动的无条件模拟(US)。我们将使用经典阻挡 实验和光遗传操作来确定美国连锁的CF的必要性和充分性 对生成CS链接的CF响应的响应。最后，在第三个目标中，我们将确定CS是否与 Cf活动推动习得的行为和小脑输出的变化。因此，我们将结合使用 光遗传学和细胞外电生理记录测试了奖赏相关的CF活动的功能。 总之，这些实验将揭示管理基于奖励的小脑学习的关键原则，以及 这种学习如何改变小脑的输出和行为。