Cerebellar circuits for reward-based learning

基于奖励的学习的小脑回路

• 批准号: 10584767
• 负责人: COURT A HULL
• 金额: $ 33.03万
• 依托单位: DUKE UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-28 至 2027-07-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10584767
• 关键词: Automobile Driving; Behavior; Behavioral; Behavioral Paradigm; Binding; Calcium; Calibration; Cerebellar Cortex; Cerebellum; Conditioned Stimulus; Cues; Data; Dendrites; Educational process of instructing; Electrodes; Evaluation; Fiber; Future; Generations; Image; Instruction; Learning; Life; Link; Measures; Mediating; Modeling; Monitor; Motor; Motor output; Movement; Mus; Outcome; Output; Pathway interactions; Play; Probability; Property; Psychological reinforcement; Purkinje Cells; Reflex action; Reporting; Response to stimulus physiology; Rewards; Signal Transduction; Site; Specificity; Stimulus; Sum; Synapses; System; Testing; Thinking; Training; Ventral Tegmental Area; Walking; Work; Writing; base; classical conditioning; dopaminergic neuron; experimental study; extracellular; fiber cell; granule cell; in vivo calcium imaging; mossy fiber; motor control; motor learning; optogenetics; postsynaptic; predictive modeling; response; reward anticipation; supervised learning; theories; transmission process; two-photon

ABSTRACT The cerebellum plays a key role in motor coordination and learning. Classic models posit that cerebellar learning is instructed by teaching signals from climbing fibers (CFs) that act according to the principles of supervised learning. While such models work well to describe CF activity and learning in some behaviors, they are not sufficient to explain CF activity in others. By developing an operant, reward-guided cerebellar-dependent task for the mouse, as well as a modified classical conditioning task, we used calcium imaging of CF input to Purkinje cell dendrites to demonstrate that CFs can be driven by reward-related task parameters. Our data suggested the possibility that CFs might engage in reinforcement learning to report predictions about expected rewards (reward prediction errors) in a similar manner as dopaminergic neurons of the ventral tegmental area (VTA). Importantly, however, our data also show significant differences from some predicitons of leading reinforcement learning models, and many other properties of cerebellar reward-based learning remain unclear. Thus, it remains largely unknown how the cerebellum operates in reward-based learning. Here will rigoursly test the hypothesis that CFs instruct cerbellar learning according to reinforcement learning rules: In the first aim, we will test whether CF activity obeys the many diverse requirements of reward prediction error signals, for example by scaling with both the probability and size of an expected reward. To do so, we will use two-photon calcium imaging to monitor CF input to Purkinje cell dendrites while manipulating reward contingencies during a classical conditioning paradigm. We will also determine the contribution of behavioral context, learning, and motor output to CF activity. In the second aim, we will test whether reward-predictive CF activity is generated by reward- responsive CF activity. This is a key property of reinforcement learning because it binds activity driven by an unconditioned simulus (US) to activity driven by a conditioned stimulus (CS). We will use classical blocking experiments and optogenetic manipulations to determine the neccessity and sufficiency of US-linked CF responses to generating CS-linked CF responses. Finally, in the third aim, we will determine whether CS-linked CF activity drives learned changes in behavior and cerebellar output. Thus, we will use a combination of optogenetics and extracellular electrophysioloigcal recordings test the function of reward-related CF activity. Together, these experiments will reveal the key principles that govern reward-based cerebellar learning, and how this learning alters cerebellar output and behavior.

摘要 小脑在运动协调和学习中起着关键作用。经典模型认为小脑的学习 通过来自攀爬纤维（CF）的教导信号来指导，该攀爬纤维根据监督的原则起作用。 学习虽然这些模型很好地描述了CF活动和某些行为中的学习，但它们不是 足以解释其他人的CF活动。通过开发一种操作性的、奖励引导的小脑依赖性任务， 对于小鼠，以及修改的经典条件反射任务，我们使用了CF输入到浦肯野的钙成像 细胞树突，以证明CFs可以由奖励相关的任务参数驱动。我们的数据表明 CF可能会参与强化学习来报告对预期奖励的预测 （奖励预测错误）以类似的方式作为腹侧被盖区（VTA）的多巴胺能神经元。 然而，重要的是，我们的数据也显示了与一些主导强化预测的显着差异 学习模型，以及小脑奖励学习的许多其他特性仍然不清楚。因此 小脑在奖励学习中是如何运作的，这在很大程度上仍不清楚。这里将严格测试 假设CFs根据强化学习规则指导cerbellar学习：在第一个目标中，我们将 测试CF活动是否服从奖励预测误差信号的许多不同要求，例如通过 与预期奖励的概率和大小进行缩放。为此，我们将使用双光子钙 成像监测CF输入到浦肯野细胞树突，同时操纵奖励应急在一个经典的 条件反射范式我们还将确定行为背景、学习和运动输出的贡献 CF活动。在第二个目标中，我们将测试奖励预测CF活动是否由奖励产生- 反应性CF活性。这是强化学习的一个关键属性，因为它绑定了由 无条件刺激（US）与条件刺激（CS）驱动的活动之间的关系。我们将使用经典的阻塞 实验和光遗传学操作来确定US连锁CF的必要性和充分性 响应以生成CS-连锁CF响应。最后，在第三个目标中，我们将确定是否与CS相关 CF活动驱动行为和小脑输出的学习变化。因此，我们将使用 光遗传学和细胞外电生理学记录测试奖赏相关CF活性的功能。 总之，这些实验将揭示控制基于奖励的小脑学习的关键原则， 这种学习如何改变小脑的输出和行为。