CAREER: The control of learning rate through multi-timescale cholinergic neuromodulation

职业：通过多时间尺度胆碱能神经调节控制学习率

• 批准号: 2145247
• 负责人: Kishore Kuchibhotla
• 金额: $ 90万
• 依托单位: Johns Hopkins University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-02-15 至 2027-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145247&HistoricalAwards=false
• 关键词: CAREER; control; learning; rate; through

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Determining how well environmental cues predict reward or punishment is critical for adaptive behavior. Past experience is more likely to be useful in stable environments. In humans and other animals, behavioral evidence suggests that learning rates depend on environmental uncertainty. In constantly changing environments, when uncertainty is high, it would be helpful to learn quickly. In stable environments, learning can be de-prioritized and instead humans and other animals can exploit their learned knowledge. This rate of learning can be formalized as a ‘learning rate’ and the computational theory of reinforcement learning (RL) aims to explain such learning processes. The proposal will test the role cholinergic neuromodulation, a deep-brain region implicated in a wide array of neurological disorders including Alzheimer’s disease (AD), in setting the learning rate during behavioral tasks. The research within this proposal is complemented with an integrated set of educational goals. A methods workshop on the optical tools that are revolutionizing neuroscience will be developed to augment an ongoing introductory neuroscience course. This workshop, for twenty students in the research track, will introduce students to optical and molecular tools. In addition, this proposal will build on the Psychological and Brain Sciences department’s goal to promote historically excluded identities through its Early Career Colloquium (ECC). A ‘Neuromodulation of Brain Circuits’ ECC segment will be launched with diverse speakers (4-6 trainees from outside JHU, 2-3 trainees within JHU and 1 keynote faculty talk) and networking events, to build a community of diverse scholars in neuromodulation. The proposed research will use quantitative behavior in mouse models and theoretical modeling to predict metalearning and then combine two-color, two-photon imaging, chemogenetics, and projection-specific optogenetics to isolate the roles of cholinergic and noradrenergic neuromodulation in setting biological learning rates. The proposal argues that the neural controller of a dynamic learning rate would benefit from three attributes: (1) encode environmental cues, (2) dynamically reflect uncertainty in the environment (i.e., high when uncertain, low when stable), and (3) modulate circuits involved in stimulus-action learning. Preliminary data show that neuromodulation of auditory cortex meets all three criteria. Cholinergic basal forebrain (CBF) axons in auditory cortex exhibit phasic, stimulus-evoked responses to auditory cues (1) that depend on preceding CBF axon activity, such that early in learning—when uncertainty is high—CBF axons ramp up their ability to discriminate the two auditory cues, and later in learning—when uncertainty is low—this discriminative signal fades (2). This CBF signal precedes cortical plasticity in a region critical for audiomotor learning (3). These data support a core hypothesis: tonic and phasic CBF signaling dynamically set the rate of cortical plasticity critical for sensorimotor learning. To test this idea, the proposal will isolate phasic, auditory input to the CBF to gain control of this signal (Goal 1), use model-based predictions to test whether CBF axon activity tracks a learning rate parameter in discrimination and reversal learning (Goal 2), and causally manipulate CBF signaling during discrimination and reversal learning and examine learning rate (Goal 3).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项全部或部分由《2021年美国救援计划法案》（公法117-2）资助。确定环境线索对奖励或惩罚的预测程度对适应性行为至关重要。过去的经验在稳定的环境中可能更有用。在人类和其他动物身上，行为证据表明，学习率取决于环境的不确定性。在不断变化的环境中，当不确定性很高时，快速学习将是有帮助的。在稳定的环境中，学习可以不被优先考虑，相反，人类和其他动物可以利用他们学到的知识。这种学习速度可以形式化为“学习率”，强化学习（RL）的计算理论旨在解释这种学习过程。该提案将测试胆碱能神经调节的作用，胆碱能神经调节与包括阿尔茨海默病（AD）在内的一系列神经系统疾病有关，在行为任务中设定学习率。该提案中的研究与一套完整的教育目标相辅相成。将开设一个关于光学工具的方法研讨会，这些工具正在革新神经科学，以增加正在进行的神经科学入门课程。本次研讨会将为20名研究方向的学生介绍光学和分子工具。此外，该提案将建立在心理和脑科学部门的目标上，即通过早期职业研讨会（ECC）促进历史上被排斥的身份。“脑回路的神经调节”ECC部分将推出不同的演讲者（4-6名JHU外部学员，2-3名JHU内部学员和1名主题教师演讲）和网络活动，以建立一个不同学者的神经调节社区。该研究将使用小鼠模型中的定量行为和理论建模来预测元学习，然后结合双色、双光子成像、化学遗传学和投影特异性光遗传学来分离胆碱能和去甲肾上腺素能神经调节在设定生物学习率中的作用。该建议认为，动态学习率的神经控制器将受益于三个属性：(1)编码环境线索，(2)动态反映环境中的不确定性（即，不确定时高，稳定时低），以及(3)参与刺激-行动学习的调制电路。初步数据表明，听觉皮层的神经调节符合所有三个标准。听觉皮层中的胆碱能基底前脑（CBF）轴突对听觉线索表现出相相的刺激诱发反应(1)，这种反应依赖于先前的CBF轴突活动，例如，在学习早期——当不确定性高时，CBF轴突增强了区分两种听觉线索的能力，而在学习后期——当不确定性低时，这种区分信号逐渐消失(2)。这个CBF信号先于听觉运动学习关键区域的皮质可塑性(3)。这些数据支持一个核心假设：强直性和阶段性CBF信号动态地设定了对感觉运动学习至关重要的皮质可塑性的速率。为了验证这一想法，该提案将分离CBF的相位，听觉输入以获得对该信号的控制（目标1），使用基于模型的预测来测试CBF轴突活动是否跟踪识别和反转学习中的学习率参数（目标2），并在区分和反转学习期间因果操纵CBF信号并检查学习率（目标3）。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。