How prefrontal cortex augments reinforcement learning

这个研究项目调查了参与人们如何学习和控制自己行为的不同大脑区域之间相互作用的本质。理论模型和经验数据表明，前额叶皮层（PFC）和基底神经节（BG）在这些类型的动机行为中相互作用，神经化学物质多巴胺在这两个大脑区域都起着核心作用。然而，很难将这些不同的大脑系统对学习的贡献分离出来。弗兰克博士和他的同事们将研究PFC如何利用其在工作记忆、认知控制、抽象和规则表示方面的作用来增强人类的强化学习。这项工作有可能大大提高我们对人类如何能够调节自己的行为作为动机和认知控制功能的理解。学习障碍在许多精神疾病中都很普遍。虽然在某些情况下，如帕金森病，所涉及的机制相对较好地理解，但在其他情况下，如精神分裂症，潜在缺陷的特征却很差。至关重要的是要适当地分离有助于学习的不同组成部分，以便将它们与导致这些缺陷的可分离机制适当地联系起来。如果将学习视为一个单一的系统，那么由于一个过程和大脑系统的损伤而产生的学习缺陷可能会被错误地归因于另一个系统，从而导致错误的结论。遵循同样的方法将揭示学习障碍的实际原因。因此，这项研究有可能确定机制，解释这种大脑回路的破坏是如何导致动机行为、认知控制和冲动障碍的。同样，发展性学习障碍可能涉及抽象和概括能力的缺陷。目的是更好地理解这些函数所需的底层机制和计算。Drs。弗兰克和柯林斯还将为STEM学科中代表性不足的女性提供计算和数据分析方法方面的指导。计算机化的实验任务将操纵被认为依赖于PFC功能的因素，包括工作记忆负荷和发现连贯规则可以用来加速学习的程度。Frank博士及其同事利用数学模型分别估算了PFC功能和基本BG过程的贡献。使用脑电图（EEG），研究人员将测量人类参与者在执行这些任务时与PFC活动相关的脑电波。脑电波活动有望预测在这些任务中的认知表现。关键的是，这种脑行为关系预计会因反映PFC和BG中多巴胺功能差异的遗传变异而有所不同。另一项研究将直接操纵多巴胺（药理学），以确定这些大脑和行为关系是如何被多巴胺水平改变的。在所有这些研究中，研究人员使用详细的数学模型，在当代理论的指导下，分离出特定的大脑-行为关系。预计遗传变异和药理学操作将影响大脑从决策结果和控制行动中学习的方式。

This research project investigates the nature of interactions between different brain regions involved in how people learn and control their actions. Theoretical models and empirical data suggest that the prefrontal cortex (PFC) and basal ganglia (BG) interact in these types of motivated behavior, and that the neurochemical dopamine plays a central role in both of these brain regions. However, it has been difficult to isolate the separable contributions of these different brain systems to learning. Dr. Frank and colleagues will investigate how PFC augments human reinforcement learning by leveraging its well-studied roles in working memory, cognitive control, abstraction, and rule representation. This work has potential to substantially advance our understanding of how humans are able to regulate their behaviors as a function of motivation and cognitive control. Learning impairments are prevalent in many psychiatric disorders. While in some cases, such as Parkinson's disease, the mechanisms involved are relatively well understood, in others, such as schizophrenia, the underlying deficits are poorly characterized. It is crucial to properly isolate different components contributing to learning, so as to appropriately relate them to separable mechanisms giving rise to those deficits. If learning is considered as a unitary system, learning deficits that arise due to impairments in one process and brain system may be mistakenly attributed to the other system and lead to erroneous conclusions. Following the same approach will shed light on the actual cause of learning impairments. As such, this research has the potential to identify mechanisms that explain how disruption of such brain circuitry leads to disorders in motivated behavior, cognitive control, and impulsivity. Similarly, developmental learning disabilities may involve a deficit in abstraction and generalization. The aim is to better understand the underlying mechanisms and computations needed for such functions. Drs. Frank and Collins will also provide mentoring on computational and data analytic methods for under-represented women in the STEM disciplines. Computerized experimental tasks will manipulate factors thought to depend on PFC function, including working memory load and the degree to which the discovery of coherent rules can be used to speed learning. Dr. Frank and colleagues use mathematical modeling to separately estimate the contributions of PFC function from that of basic BG processes. Using electroencephalography (EEG), the investigators will measure human participants' brain waves associated with PFC activity while they perform these tasks. Brain wave activity is expected to predict cognitive performance on these tasks. Critically, this brain-behavior relationship is expected to differ as a function of genetic variants that reflect differences in dopamine function in PFC and BG. Another study will directly manipulate dopamine (pharmacologically) in order to determine how these brain and behavior relationships are causally altered by dopamine levels. In all of these studies the investigators use detailed mathematical models to isolate specific brain-behavior relationships guided by contemporary theory. It is expected that genetic variants and pharmacological manipulations will affect the way that the brain learns from decision outcomes and controls actions.