fMRI investigations of how we learn what is relevant for a decision

描述(申请人提供)：药物滥用是一种与大脑中奖赏学习和决策机制的妥协有关的障碍，特别是中脑多巴胺系统及其纹状体目标。著名的理论表明，滥用药物与多巴胺相互作用，“劫持”正常的学习，而不是引导行为指向药物的采购和消费。近年来，强化学习的计算框架在理解多巴胺在奖赏学习和决策中的作用，以及其信号的轻微改变如何导致这种有害影响方面取得了长足的进步。决策的强化学习模型描述了基底节结构如何根据不同刺激的未来奖励价值来学习评估不同的刺激，以及多巴胺能活动如何影响这种学习过程。但是，在所有可用的刺激中，大脑如何识别哪些是代表和评估的相关刺激呢？这一“表征学习”问题在实验和计算文献中基本上都被忽视了，可能是物质滥用障碍的核心。物质滥用者并不完全是非理性的决策者；事实上，研究表明，他们在购买滥用药物时采用了正常的经济决策。然而，一个明显的反常现象是，他们执着于预测毒品奖励的刺激(以及他们有时在学习操纵这些刺激以获得毒品方面的巨大创造力)，而不考虑替代奖励的预测因素，如工作工资和家庭支持。这一建议的动机是这样一种假设，即这种倾斜的注意力可能是异常表征学习过程的结果，该过程导致对药物奖励预测线索的过度表征。这项提议的目标是对表征学习的计算和神经基础以及它与人脑中的奖励学习的相互作用进行行为和功能磁共振研究。这些研究将采用一种专门为突出这种相互作用而设计的新颖的决策任务。要检验的假设是，前额叶皮质构建世界的表征，识别并将注意力引导到与手头任务相关的刺激维度，并构建可供基底神经节在强化学习过程中使用的表征。此外，我们假设多巴胺能回路和相关的预测误差信号在前额叶皮质的表征学习和基底神经节的强化学习之间的相互作用中起中介作用。这项研究将通过检测与表征学习相关的神经信号，并揭示人类进行表征学习的计算策略，来初步检验这些假说。了解表征学习过程，它们在神经回路中的实现，以及它们如何受到药物敏感的神经调节剂(如多巴胺)的影响，将为药物影响的决策过程中的错误提供理论依据，并为药物滥用的诊断和治疗提供新的方向。 与公共健康相关：药物滥用是一个严重的公共健康问题，其核心是大脑获得奖励的决策机制。虽然许多研究集中在滥用药物如何改变奖赏学习机制上，但很少有人关注更基本(或许也更脆弱)的学习过程，即在所有可用的刺激中，哪些与决策相关，应该得到关注和了解。这个项目计划研究这种“表征学习”过程，以及它如何与奖赏学习相互作用，无论是在神经上还是在计算上。希望更好地理解表征学习策略及其神经实现将使我们能够确定这些策略是如何受到药物滥用的影响的，并将有助于开发针对这些额外方面的障碍的新治疗策略。

DESCRIPTION (provided by applicant): Substance abuse is a disorder that has been associated with a compromise of reward learning and decision making mechanisms in the brain, notably, the midbrain dopamine system and its striatal targets. Prominent theories suggest that drugs of abuse interact with dopamine, "hijacking" normal learning and instead directing behavior towards the procurement and consumption of the drug. In recent years, the computational framework of reinforcement learning has been leveraged to make great strides in understanding the role of dopamine in reward learning and decision making, and how slight modifications of its signals could lead to such detrimental effects. Reinforcement learning models of decision making describe how basal-ganglia structures learn to evaluate different stimuli in terms of their future reward value, and how dopaminergic activity affects such learning processes. But how does the brain identify which, of all the available stimuli, are the relevant ones to represent and evaluate? This "representation learning" problem has been largely ignored in both the experimental and the computational literature, and may lie at the heart of substance abuse disorder. Substance abusers are not wholly irrational decision makers; in fact, research shows that they employ normal economic decision making to the purchase of drugs of abuse. Nevertheless, a clear abnormality is their fixation on stimuli predicting drug rewards (and their sometimes great creativity in learning to manipulate these to obtain drugs) to the exclusion of consideration of predictors of alternative rewards such as salary from holding a job and the support of one's family. This proposal is motivated by the hypothesis that this skewed attention may be the result of an abnormal representation learning process that causes an over-representation of drug-reward predicting cues. The goal of this proposal is to carry out behavioral and fMRI investigations of the computational and neural basis of representation learning and its interaction with reward learning in the human brain. The studies will employ a novel decision making task that has been specifically designed to highlight this interaction. The hypothesis to be tested is that the prefrontal cortex constructs representations of the world, identifying and directing attention to stimulus dimensions that are relevant for the task at hand, and constructing representations that can be used by the basal ganglia in the process of reinforcement learning. Moreover, we hypothesize that dopaminergic circuitry and related prediction error signals mediate the interaction between representation learning in the prefrontal cortex and reinforcement learning in the basal ganglia. The research proposed will provide initial testing of these hypotheses by detecting neural signals related to representation learning and uncovering the computational strategies by which representation learning proceeds in humans. Understanding representation learning processes, their realization in neural circuitry, and how they are influenced by drug-sensitive neuromodulators such as dopamine, will inform theories of what it is that goes awry in drug-influenced decision making, and provide new directions for diagnosis and treatment of substance abuse. PUBLIC HEALTH RELEVANCE: Substance abuse is a serious problem of public health that centers around the brain's decision-making mechanisms for obtaining rewards. While much research has concentrated on how drugs of abuse might alter reward learning mechanisms, little attention has been devoted to the more fundamental (and perhaps more fragile) process of learning which of all the available stimuli are relevant to a decision and should be attended to and learned about. This project proposes to study this "representation learning" process and how it interacts with reward learning, both neurally and computationally. It is hoped that better understanding of representation learning strategies and their neural implementation will allow us to identify how these are affected by drugs of abuse, and will help in developing new treatment strategies targeted at these additional aspects of the disorder.