Determining the explanatory utility of computational reinforcement-learning theories of goal-directed and habitual control at behavioral and neural levels

确定计算强化学习的解释效用 行为和神经层面的目标导向和习惯控制理论 派：约翰·P·奥多尔蒂医生 院校：加州理工学院 项目总结 越来越多的证据支持大脑中存在两种不同的指导行动选择的系统： 一种以目标为导向的系统，在该系统中，根据 相关的目标或结果，以及一种习惯系统，在这种系统中，行动的选择是条件反射的，完全基于 他们过去增援的历史。这两个系统的计算帐户已在#年制定。 计算强化学习(RL)理论两种不同变体的术语：基于模型的(MB)VS 无模型(MF)RL。然而，支持拟议的两者之间对应关系的经验证据 心理(RDoC级别)和计算性RL帐户是稀疏的。在这里，我们的目标是全面 目标导向和习惯控制的RDoC水平结构能否被有效地描述 通过基于模型的和无模型的人类RL在行为和神经上的计算框架 级别。 我们计划管理两个不同的行为任务，旨在区分目标导向和习惯性 对照和MB从MF对照到一大群健康参与者(n=200)和未分化的队列 精神病人(n=100)。我们的参与者将在接受fMRI扫描的同时执行这些任务，在 除进行静息功能磁共振成像外，还进行弥散加权成像。我们还将测量行为 同一个体与精神病理学相关的特征和状态。我们将利用个体差异 通过我们的行为、计算和神经测量来确定 心理构思和计算账户最好被视为一回事，或者是 相比之下，它们在理论上的重要方面存在分歧。我们是否应该发现两者之间的明显差异 心理学(RDoC)结构和计算描述，在我们使用的任何分析水平上，这 将促使计算框架的迭代改进，以更好地近似心理 (RDoC)级别构建，与实验目标并行完成。两者之间的区别 目标和习惯及其提出的计算基础可以说是最有影响力的研究之一 到目前为止计算精神病学的主题，考虑到这些概念作为一种手段的假设相关性 捕捉到各种形式的精神障碍。因此，更好地理解 这些结构之间的关系，再加上计算理论的积极完善过程 要实现与心理结构的更紧密的对应，将是在 这个域。

Determining the explanatory utility of computational reinforcement-learning theories of goal-directed and habitual control at behavioral and neural levels PI: Dr. John P. O’Doherty Institution: California Institute of Technology PROJECT SUMMARY Accumulating evidence supports the existence of two distinct systems for guiding action-selection in the brain: a goal-directed system in which actions are selected with reference to the current incentive value of the associated goal or outcome, and a habitual system in which actions are selected reflexively, based solely on their history of past reinforcement. A computational account for these two systems has been formulated in terms of two distinct variants of computational reinforcement-learning (RL) theory: model-based (MB) vs model-free (MF) RL. Yet, empirical evidence in support of the proposed correspondence between the psychological (RDoC level) and computational RL accounts are sparse. Here we aim to comprehensively address whether the RDoC level constructs of goal-directed and habitual control can be effectively described by the computational framework of model-based and model-free RL in humans at both behavioral and neural levels. We plan to administer two distinct behavioral tasks designed to discriminate goal-directed from habitual control and MB from MF control to a large cohort of healthy participants (n=200) and an undifferentiated cohort of psychiatric patients (n=100). Our participants will perform these tasks while being scanned with fMRI, in addition to undergoing resting-state fMRI, and diffusion weighted imaging. We will also measure behavioral traits and states relevant to psychopathology in the same individuals. We will leverage individual differences across our behavioral, computational and neural measures in order to determine the extent to which the psychological constructs and computational accounts are best viewed as being one and the same, or whether by contrast they diverge in theoretically important ways. Should we detect clear differences between the psychological (RDoC) constructs and computational descriptions on any of the levels of analysis we utilize, this will motivate an iterative refinement of the computational framework to better approximate the psychological (RDoC) level constructs, to be accomplished in parallel to the experimental aims. The distinction between goals and habits and their proposed computational bases are arguably one of the most influential research topics in computational psychiatry to date, given the hypothesized relevance of these constructs as a means of capturing various forms of psychiatric dysfunction. Thus, a better understanding of the nature of the relationship between these constructs, coupled with a process of active refinement of the computational theory to achieve a much closer correspondence to the psychological constructs, is going to be critical for progress in this domain.