CAREER: The control of learning rate through multi-timescale cholinergic neuromodulation

该奖项的全部或部分资金来自《2021年美国救援计划法案》(公法117-2)。确定环境线索预测奖惩的程度对适应行为至关重要。过去的经验在稳定的环境中更有可能是有用的。在人类和其他动物中，行为证据表明，学习速度取决于环境的不确定性。在不断变化的环境中，当不确定性很高时，快速学习将是有帮助的。在稳定的环境中，学习可以被排在次要位置，取而代之的是，人类和其他动物可以利用他们学到的知识。这种学习速率可以被形式化地描述为“学习速率”，强化学习的计算理论(RL)旨在解释这种学习过程。该提案将测试胆碱能神经调节在设定行为任务中学习速度方面的作用。胆碱能神经调节是大脑深层区域，与包括阿尔茨海默病(AD)在内的一系列神经疾病有关。这项建议中的研究与一套综合的教育目标相辅相成。将开发一个关于正在给神经科学带来革命性变化的光学工具的方法研讨会，以补充正在进行的神经科学入门课程。本次研讨会面向20名处于研究轨道的学生，将向学生介绍光学和分子工具。此外，这项提议将建立在心理和脑科学系通过其早期职业座谈会(ECC)促进历史上被排除在外的身份的目标的基础上。将推出“大脑回路的神经调节”ECC环节，有不同的演讲者(4-6名来自JHU的学员，2-3名JHU内的学员和1个主旨教师讲座)和网络活动，以建立一个神经调节领域的不同学者社区。这项拟议的研究将使用小鼠模型中的量化行为和理论建模来预测元学习，然后结合双色、双光子成像、化学遗传学和投影特异性光遗传学来分离胆碱能和去甲肾上腺素能神经调节在设定生物学习速率中的作用。该建议认为，动态学习速率的神经控制器将受益于三个属性：(1)编码环境线索，(2)动态反映环境中的不确定性(即，不确定时高，稳定时低)，以及(3)调制刺激-动作学习中涉及的电路。初步数据显示，听觉皮质的神经调节满足所有三个标准。听觉皮质中的胆碱能基底前脑(CBF)轴突对听觉线索表现出阶段性的刺激诱发反应(1)，这取决于先前的CBF轴突活动，因此在学习的早期-当不确定性较高时-CBF轴突增强了它们区分两个听觉线索的能力，而在学习的后期-当不确定性较低-这种辨别信号消失(2)。在听觉运动学习的关键区域，这种CBF信号先于皮质的可塑性(3)。这些数据支持一个核心假说：强直和时相CBF信号动态设定皮层可塑性的速率，对感觉运动学习至关重要。为了测试这一想法，该提案将分离CBF的阶段性、听觉输入以获得对该信号的控制(目标1)，使用基于模型的预测来测试CBF轴突活动是否跟踪辨别和反转学习中的学习速率参数(目标2)，并在辨别和反转学习期间因果操纵CBF信号并检查学习率(目标3)。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Determining how well environmental cues predict reward or punishment is critical for adaptive behavior. Past experience is more likely to be useful in stable environments. In humans and other animals, behavioral evidence suggests that learning rates depend on environmental uncertainty. In constantly changing environments, when uncertainty is high, it would be helpful to learn quickly. In stable environments, learning can be de-prioritized and instead humans and other animals can exploit their learned knowledge. This rate of learning can be formalized as a ‘learning rate’ and the computational theory of reinforcement learning (RL) aims to explain such learning processes. The proposal will test the role cholinergic neuromodulation, a deep-brain region implicated in a wide array of neurological disorders including Alzheimer’s disease (AD), in setting the learning rate during behavioral tasks. The research within this proposal is complemented with an integrated set of educational goals. A methods workshop on the optical tools that are revolutionizing neuroscience will be developed to augment an ongoing introductory neuroscience course. This workshop, for twenty students in the research track, will introduce students to optical and molecular tools. In addition, this proposal will build on the Psychological and Brain Sciences department’s goal to promote historically excluded identities through its Early Career Colloquium (ECC). A ‘Neuromodulation of Brain Circuits’ ECC segment will be launched with diverse speakers (4-6 trainees from outside JHU, 2-3 trainees within JHU and 1 keynote faculty talk) and networking events, to build a community of diverse scholars in neuromodulation. The proposed research will use quantitative behavior in mouse models and theoretical modeling to predict metalearning and then combine two-color, two-photon imaging, chemogenetics, and projection-specific optogenetics to isolate the roles of cholinergic and noradrenergic neuromodulation in setting biological learning rates. The proposal argues that the neural controller of a dynamic learning rate would benefit from three attributes: (1) encode environmental cues, (2) dynamically reflect uncertainty in the environment (i.e., high when uncertain, low when stable), and (3) modulate circuits involved in stimulus-action learning. Preliminary data show that neuromodulation of auditory cortex meets all three criteria. Cholinergic basal forebrain (CBF) axons in auditory cortex exhibit phasic, stimulus-evoked responses to auditory cues (1) that depend on preceding CBF axon activity, such that early in learning—when uncertainty is high—CBF axons ramp up their ability to discriminate the two auditory cues, and later in learning—when uncertainty is low—this discriminative signal fades (2). This CBF signal precedes cortical plasticity in a region critical for audiomotor learning (3). These data support a core hypothesis: tonic and phasic CBF signaling dynamically set the rate of cortical plasticity critical for sensorimotor learning. To test this idea, the proposal will isolate phasic, auditory input to the CBF to gain control of this signal (Goal 1), use model-based predictions to test whether CBF axon activity tracks a learning rate parameter in discrimination and reversal learning (Goal 2), and causally manipulate CBF signaling during discrimination and reversal learning and examine learning rate (Goal 3).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.