Computational mechanisms of goal-directed control

描述（由申请人提供）：人类和动物如何做出决定以及为了奖励而做出决定一直是人们关注的焦点。这些问题之所以引人注目，一方面是因为它们与现实世界的问题（从日常购买到经济政策）有着关键的相关性，另一方面是因为它们与从帕金森症到精神分裂症等神经递质系统潜在疾病的关系。许多关于决策的当代研究都是由计算模型的发展推动的，这些模型对决策的产生做出了具体的预测。这些基于强化学习理论的模型，为梳理决策的认知和生理机制提供了宝贵的工具。然而，这些模型迄今为止只适用于习惯性决策，即那些由于学习期望特定行为产生特定结果而产生的决策。现实世界的决策还包括另一类决策，这类决策包括在你对潜在行为的结果有任何经验的情况下进行计划。当做出非习惯性的决定时，个体可能会使用他们最初在没有任何回报的情况下学到的信息。例如，我们选择尝试新菜或在全新的餐馆吃饭，即使我们以前可能从未进入过它们。对这类行为进行建模已被证明是极其困难的，部分原因是可能对此类决策产生影响的信息种类繁多。最近，我们开发了一种简化的、受限的实验学习任务，使我们能够同时分别测量人类的习得习惯和非习惯学习。我们已经建立了第二种学习形式的模型，并使用功能性磁共振成像（fMRI），确定了代表学习信息的神经结构。其中包括海马体，这是一种对正常记忆至关重要的结构，其功能障碍与几种主要的精神健康障碍有关，如重度抑郁症和精神分裂症。然而，海马体在奖励决策中的位置尚不清楚。这个建议建立在我们之前的研究结果的基础上，通过要求参与者将这些信息应用于赚钱，来确定这些信息是如何用于决策的。具体来说，我们检查已知参与决策的大脑系统，并询问他们使用什么方法来解析他们现在可以获得的信息。我们有理由相信，这些系统采用策略来减少它们需要处理的信息量，而海马体具有执行这些策略的独特能力。了解这些策略对于理解在现实世界中如何做出决策至关重要，并将为海马体功能的基本机制提供有价值的和新颖的见解。

DESCRIPTION (provided by applicant): How humans and animals make decisions and decisions for rewards have been a subject of intense focus recently. These questions are compelling both for their critical relevance to real-world concerns, from daily purchases to economic policy, and their relationship to neurotransmitter systems underlying diseases from Parkinson's to schizophrenia. Much contemporary study of decisions has been spurred by the development of computational models that make specific predictions about how decisions arise. These models, based on the theories of reinforcement learning, have provided an invaluable tool for teasing apart the cognitive and physiological mechanisms of decision-making. However these models have to date only been applied to habitual decisions - those decisions that result from learning to expect a particular outcome from a particular action. Real-world decision- making also encompasses another class of decisions, which involve planning in spite of any experience with the outcome of your potential actions. When making non-habitual decisions, individuals may use information that they originally learned without any reward. For instance, we choose to sample new dishes or eat at entirely new restaurants even though we may have never before entered them. Modeling this sort of behavior has proven extremely difficult, due in part to the wide variety of information that may be brought to bear on such decisions. Recently, we have developed a reduced, constrained experimental learning task that allows us to separately measure both learned habits and non-habitual learning, simultaneously, in humans. We have modeled this second form of learning, and, using functional magnetic resonance imaging (fMRI), identified neural structures that represent the learned information. These include the hippocampus, a structure critical for normal memory, and whose dysfunction is implicated in several major mental health disorders, such as major depression and schizophrenia. The place of the hippocampus in decisions for reward is, however, unclear. This proposal builds on our previous results to identify how this information is used to make decisions, by asking participants to apply this information to making money. Specifically, we examine brain systems known to participate in decision-making, and ask what methods they use to parse through the information now available to them. We have reason to believe that these systems employ strategies to reduce the amount of information they need to work with, and that hippocampus is uniquely capable of implementing these strategies. Understanding these strategies is essential to understanding how decisions are made in the real world, and will provide valuable and novel insight into the fundamental mechanisms of hippocampal function. PUBLIC HEALTH RELEVANCE: I aim to elucidate the mechanisms by which hippocampus interacts with striatal and cortical decision structures to effect goal-directed planning behavior. This work is relevant to public health as functional and structural hippocampal deficits are strongly associated with numerous severe mental health disorders. In particular, several of these disorders - for example, schizophrenia and major depression - exhibit core symptoms which reflect dysfunction of exactly the sorts of associative learning mechanisms proposed to underlie goal-directed decisions. An understanding of these mechanisms will thus provide crucial insights into the nature and extent of such disruptions and inform increasingly sophisticated and targeted development of behavioral and physiological therapies.