fMRI investigations of how we learn what is relevant for a decision

描述（由申请人提供）：药物滥用是一种与大脑中奖励学习和决策机制受损有关的疾病，特别是中脑多巴胺系统及其纹状体目标。著名的理论认为，滥用药物与多巴胺相互作用，“劫持”了正常的学习，而不是将行为导向药物的获取和消耗。近年来，强化学习的计算框架在理解多巴胺在奖励学习和决策中的作用以及其信号的轻微修改如何导致这种有害影响方面取得了巨大进展。决策的强化学习模型描述了基底神经节结构如何根据未来的奖励价值来学习评估不同的刺激，以及多巴胺能活动如何影响这种学习过程。但是，大脑如何在所有可用的刺激中识别哪些是需要表征和评估的相关刺激呢？这种“表征学习”问题在实验和计算文献中都被很大程度上忽视了，而且可能是药物滥用障碍的核心。药物滥用者并非完全非理性的决策者；事实上，研究表明，他们使用正常的经济决策来购买滥用药物。然而，一个明显的异常是，他们对预测药物奖励的刺激（以及他们有时在学习操纵这些刺激以获得药物方面的巨大创造力）的关注，而排除了对其他奖励的预测因素的考虑，比如保住工作的薪水和家庭的支持。这一提议的动机是这样一种假设，即这种扭曲的注意力可能是一个异常的表征学习过程的结果，该过程导致了对药物奖励预测线索的过度表征。本提案的目标是对人脑中表征学习的计算和神经基础及其与奖励学习的相互作用进行行为和功能磁共振成像研究。这些研究将采用一种新的决策任务，该任务是专门为突出这种相互作用而设计的。待测试的假设是，前额叶皮层构建世界表征，识别并引导注意力到与手头任务相关的刺激维度，并构建可以在强化学习过程中被基底神经节使用的表征。此外，我们假设多巴胺能回路和相关的预测误差信号介导了前额叶皮层的表征学习和基底神经节的强化学习之间的相互作用。提出的研究将通过检测与表征学习相关的神经信号和揭示人类表征学习进行的计算策略，为这些假设提供初步测试。理解表征学习过程，它们在神经回路中的实现，以及它们如何受到药物敏感的神经调节剂（如多巴胺）的影响，将为药物影响下的决策出错提供理论依据，并为药物滥用的诊断和治疗提供新的方向。

DESCRIPTION (provided by applicant): Substance abuse is a disorder that has been associated with a compromise of reward learning and decision making mechanisms in the brain, notably, the midbrain dopamine system and its striatal targets. Prominent theories suggest that drugs of abuse interact with dopamine, "hijacking" normal learning and instead directing behavior towards the procurement and consumption of the drug. In recent years, the computational framework of reinforcement learning has been leveraged to make great strides in understanding the role of dopamine in reward learning and decision making, and how slight modifications of its signals could lead to such detrimental effects. Reinforcement learning models of decision making describe how basal-ganglia structures learn to evaluate different stimuli in terms of their future reward value, and how dopaminergic activity affects such learning processes. But how does the brain identify which, of all the available stimuli, are the relevant ones to represent and evaluate? This "representation learning" problem has been largely ignored in both the experimental and the computational literature, and may lie at the heart of substance abuse disorder. Substance abusers are not wholly irrational decision makers; in fact, research shows that they employ normal economic decision making to the purchase of drugs of abuse. Nevertheless, a clear abnormality is their fixation on stimuli predicting drug rewards (and their sometimes great creativity in learning to manipulate these to obtain drugs) to the exclusion of consideration of predictors of alternative rewards such as salary from holding a job and the support of one's family. This proposal is motivated by the hypothesis that this skewed attention may be the result of an abnormal representation learning process that causes an over-representation of drug-reward predicting cues. The goal of this proposal is to carry out behavioral and fMRI investigations of the computational and neural basis of representation learning and its interaction with reward learning in the human brain. The studies will employ a novel decision making task that has been specifically designed to highlight this interaction. The hypothesis to be tested is that the prefrontal cortex constructs representations of the world, identifying and directing attention to stimulus dimensions that are relevant for the task at hand, and constructing representations that can be used by the basal ganglia in the process of reinforcement learning. Moreover, we hypothesize that dopaminergic circuitry and related prediction error signals mediate the interaction between representation learning in the prefrontal cortex and reinforcement learning in the basal ganglia. The research proposed will provide initial testing of these hypotheses by detecting neural signals related to representation learning and uncovering the computational strategies by which representation learning proceeds in humans. Understanding representation learning processes, their realization in neural circuitry, and how they are influenced by drug-sensitive neuromodulators such as dopamine, will inform theories of what it is that goes awry in drug-influenced decision making, and provide new directions for diagnosis and treatment of substance abuse. PUBLIC HEALTH RELEVANCE: Substance abuse is a serious problem of public health that centers around the brain's decision-making mechanisms for obtaining rewards. While much research has concentrated on how drugs of abuse might alter reward learning mechanisms, little attention has been devoted to the more fundamental (and perhaps more fragile) process of learning which of all the available stimuli are relevant to a decision and should be attended to and learned about. This project proposes to study this "representation learning" process and how it interacts with reward learning, both neurally and computationally. It is hoped that better understanding of representation learning strategies and their neural implementation will allow us to identify how these are affected by drugs of abuse, and will help in developing new treatment strategies targeted at these additional aspects of the disorder.