Neural mechanisms of cognitive control: interactions of goal-directed and habit systems

我研究的总体目标是了解大脑如何在复杂和动态的环境中适应选择，以实现长期目标。哺乳动物的大脑至少有两个不同的系统可以控制选择：一个是利用奖励结果快速适应选择的“目标导向”系统；另一个是涉及缓慢适应选择的感觉运动处理的“习惯”系统。我的NSERC资助的工作导致了一个重要的发现，即习惯系统本身就可以驱动对忽略奖励后的选择的快速适应。这与大多数流行的选择理论相矛盾，并表明需要进行重大修改。此外，这一发现为研究目标导向系统如何抑制习惯系统提供了一种新的方法。这种“认知控制”对于通过抑制短期收益的行为来实现长期目标很重要，比如为了减肥而放弃甜点。在接下来的5年里，我的研究计划的近期目标是利用这些发现来促进我们对目标导向系统调节习惯的神经机制的理解，并找出为什么在任务不确定性期间，这种机制在性别之间往往是不同的。具体地说，我们将(1)使用大鼠的神经记录来识别选择过程中习惯和目标系统中奖励、不确定性和反应抑制的神经编码，(2)研究通过增加任务不确定性来促进目标导向控制如何影响这些系统的神经编码和相互作用，(3)调查生物性别对这些过程的影响，以及(4)调查系统之间基于不确定性的竞争的计算优势。这是增进我们对几种普遍的行为现象的认识所需的基本信息，包括试错式学习、对选择的认知控制以及涉及不确定结果的选择范式中的性别差异(例如赌博)。我们将利用这样一个事实，即大鼠和人类在我们的新选择任务中表现非常相似，这使得我们能够将啮齿动物的神经记录转化为人类的选择表现。我们将使用计算技术和分析来估计对隐藏变量(例如学习率)的处理效果，并测试受实验结果启发的算法的计算性能。这个研究项目支持了我的长期目标，即开发一种机械论理论，研究大脑如何在目标导向系统中使用环境(信念)的心理模型来破坏习惯性反应，这对适应许多现实世界中的行为至关重要。我的多学科研究计划整合了动物、人类和计算模型方面的工作，并为HQP提供了特殊的培训。据我所知，这个研究项目具有很高的原创性，有望在我们的大脑决策理论以及我们如何设计智能人工系统方面取得革命性的进展。

The overall goal of my research is to understand how the brain adapts choices in complex and dynamic environments to achieve long-term goals. The mammalian brain has at least two distinct systems that can control choice: a `goal directed' system that uses reward outcomes to adapt choices quickly; and a `habit' system involving sensorimotor processing that adapts choices slowly. My NSERC-funded work led to the important discovery that the habit system alone can drive rapid adaptation of choice following omission of reward. This contradicts most prevailing theories of choice, and indicates that major revisions are needed. Moreover, this finding reveals a novel method to study how the goal-directed system can suppress the habit system. Such `cognitive control' is important for achieving long-term goals by suppressing actions for short-term gain, such as foregoing dessert to lose weight. The near-term objectives of my research program over the next 5 years are to capitalize on these discoveries to advance our understanding of the neural mechanisms by which goal-directed systems can regulate habits, and to identify why this is often different among sexes during task uncertainty. Specifically, we will (1) use neural recordings in rats to identify neural encoding of reward, uncertainty, and response inhibition in habit and goal systems during choice, (2) examine how promoting goal-directed control by increasing task uncertainty affects neural encoding and interactions of these systems, (3) investigate the effects of biological sex on these processes, and (4) investigate the computational advantages of uncertainty-based competition between systems. This is essential information needed for advancing our knowledge of several prevalent behavioural phenomena, including trial-and-error learning, cognitive control of choice, and sex differences in choice paradigms involving uncertain outcomes (e.g. gambling). We will take advantage of the fact that rats and humans perform very similarly on our novel choice task, which allows us to translate neural recordings in behaving rodents to choice performance in humans. We will use computational techniques and analytics to estimate treatment effects on hidden variables (e.g. learning rates) and test the computational properties of algorithms inspired by the experimental results. This program of research supports my long-term objective of developing a mechanistic theory for how the brain uses mental models of the environment (beliefs) in goal-directed systems to disrupt habitual responses, which is of fundamental importance for adapting behaviour in many real-world situation. My multidisciplinary research program integrates work in animals, humans, and computational models, and provides exceptional training for HQP. As far as I am aware, this research program is highly original and is poised for transformative advances in our theory of decision-making in the brain and how we might engineer intelligent artificial systems.