Computational Substrates of Addiction and Reward

6.项目总结/摘要 滥用药物的部分运作方式是破坏适应机制（算法）， 通常用于评估行为和做出决策。众所周知，如果不是 大多数药物滥用影响中脑多巴胺系统，近年来详细 多巴胺能功能的计算模型（称为强化学习或RL 模型）已经发展到现在可以用来解释 功能成像实验对人类受试者的奖励学习。这些相同 模型也可以解释成瘾状态的重要特征。RL模型预测 包括期望在内的一系列学习信号的存在和行为影响 错误（预期和实际结果之间的持续差异）和虚构 错误（“可能发生的事情”与实际结果之间的持续差异）。 因此，RL模型与多巴胺系统的连接立即 建议使用它们作为学习和决策的定量探针， 成瘾人群尽管RL模型之间有着密切的联系， 多巴胺系统，奖励引导的选择，没有模型驱动的成像方法 已经被用来探测任何上瘾的人群。在这份提案中，我们试图填补这一空白。 差距，并将追求一个严格的，基于模型的方法，以奖励依赖学习 信号，它们的产生，以及它们在人类经历中的数学特征 功能磁共振成像，同时他们执行顺序选择任务。这 我们将在健康的人类受试者和吸烟者中进行研究， 开发了大量的初步数据，以支持这一具体目标， 项目我们选择将基于模型的方法应用于吸烟者，因为 使用法律的药物，滥用多种物质的流行率较低，吸烟一般是 被认为是其他药物滥用的入门药物，吸烟者代表了一个很大的 对社会造成健康负担，特别是在晚年。通过使用RL模型来指导 设计，分析和解释一系列奖励收获实验，这 该提案将对奖励的计算基础产生新的见解- 依赖性的选择和它的病理劫持一种常见的滥用药物。

6. PROJECT SUMMARY/ ABSTRACT Drugs of abuse operate in part by subverting the adaptive mechanisms (algorithms) that are normally used to value actions and make decisions. It is well known that many if not most drugs of abuse affect midbrain dopamine systems, and in recent years detailed computational models of dopaminergic function (called Reinforcement Learning or RL models) have advanced to the point that they are now used profitably to interpret functional imaging experiments on reward learning in human subjects. These same models can also account for important features of the addicted state. RL models predict the existence and behavioral impact of a range of learning signals including expectation errors (ongoing differences between expectations and actual outcomes) and fictive errors (ongoing differences between `what might have happened' and actual outcomes). Consequently, the connection of RL models to dopamine systems immediately recommends their use as quantitative probes of learning and decision-making in addicted populations. Despite the intimate connection between RL models, midbrain dopamine systems, and reward-guided choice, no model-driven imaging approaches have been used to probe any addicted populations. In this proposal, we seek to fill this gap, and will pursue a rigorous, model-based approach to reward-dependent learning signals, their generation, and their mathematical character in humans undergoing functional magnetic resonance imaging while they execute sequential choice tasks. This effort will be carried out in healthy human subjects and smokers, and we have developed a substantial body of preliminary data to support the specific goals of this project. We have chosen to apply a model-based approach to smokers because they use a legal drug, there is less prevalence of poly-substance abuse, smoking is generally thought to be a gateway drug for other drugs of abuse, and smokers represent a large health burden on society especially in their later years. By using RL models to guide the design, analysis, and interpretation of a range of reward-harvesting experiments, this proposal will yield new insights into the computational underpinnings of reward- dependent choice and its pathological hijacking by a common drug of abuse.