Determining the explanatory utility of computational reinforcement-learning theories of goal-directed and habitual control at behavioral and neural levels

确定计算重复学习的解释效用 行为和神经层面的目标导向和习惯控制理论 PI：John P. O'Doherty博士 院校：加州理工学院 项目摘要 越来越多的证据支持大脑中存在两个不同的指导行动选择的系统： 一个目标导向系统，在该系统中，参考当前的激励值来选择行动， 相关的目标或结果，以及一个习惯性的系统，在这个系统中，行动被反射性地选择，仅仅基于 他们过去的强化历史。这两个系统的计算帐户已制定在 计算重复学习（RL）理论的两种不同变体的术语：基于模型（MB）与 无模型（MF）RL。然而，支持所提出的对应关系的经验证据表明， 心理（RDoC级别）和计算RL帐户是稀疏的。在这里，我们的目标是全面 解决是否可以有效地描述目标导向和习惯控制的RDoC级别结构 基于模型和无模型的RL在人类行为和神经上的计算框架 程度. 我们计划管理两个不同的行为任务，旨在区分目标导向和习惯性 对照组和MF对照组的MB至健康受试者的大队列（n=200）和未分化队列 精神病患者（n=100）。我们的参与者将在接受功能磁共振成像扫描的同时执行这些任务， 除了进行静息态功能磁共振成像和弥散加权成像。我们还将测量行为 与精神病理学相关的特征和状态。我们将利用个体差异 在我们的行为，计算和神经措施，以确定在多大程度上， 心理构造和计算账户最好被看作是同一个，或者 相比之下，它们在理论上的重要方面存在分歧。我们是否应该发现 心理（RDoC）结构和计算描述的任何水平的分析，我们利用，这 将激励计算框架的迭代细化，以更好地近似心理 （RDoC）水平的结构，要完成平行的实验目标。之间的区别 目标和习惯及其提出的计算基础可以说是最有影响力的研究之一， 迄今为止，计算精神病学的主题，假设这些结构的相关性作为一种手段， 捕捉各种形式的精神障碍。因此，更好地了解 这些结构之间的关系，再加上积极完善的计算理论的过程 为了实现更接近的对应心理结构，将是至关重要的进展， 这个域。

Determining the explanatory utility of computational reinforcement-learning theories of goal-directed and habitual control at behavioral and neural levels PI: Dr. John P. O’Doherty Institution: California Institute of Technology PROJECT SUMMARY Accumulating evidence supports the existence of two distinct systems for guiding action-selection in the brain: a goal-directed system in which actions are selected with reference to the current incentive value of the associated goal or outcome, and a habitual system in which actions are selected reflexively, based solely on their history of past reinforcement. A computational account for these two systems has been formulated in terms of two distinct variants of computational reinforcement-learning (RL) theory: model-based (MB) vs model-free (MF) RL. Yet, empirical evidence in support of the proposed correspondence between the psychological (RDoC level) and computational RL accounts are sparse. Here we aim to comprehensively address whether the RDoC level constructs of goal-directed and habitual control can be effectively described by the computational framework of model-based and model-free RL in humans at both behavioral and neural levels. We plan to administer two distinct behavioral tasks designed to discriminate goal-directed from habitual control and MB from MF control to a large cohort of healthy participants (n=200) and an undifferentiated cohort of psychiatric patients (n=100). Our participants will perform these tasks while being scanned with fMRI, in addition to undergoing resting-state fMRI, and diffusion weighted imaging. We will also measure behavioral traits and states relevant to psychopathology in the same individuals. We will leverage individual differences across our behavioral, computational and neural measures in order to determine the extent to which the psychological constructs and computational accounts are best viewed as being one and the same, or whether by contrast they diverge in theoretically important ways. Should we detect clear differences between the psychological (RDoC) constructs and computational descriptions on any of the levels of analysis we utilize, this will motivate an iterative refinement of the computational framework to better approximate the psychological (RDoC) level constructs, to be accomplished in parallel to the experimental aims. The distinction between goals and habits and their proposed computational bases are arguably one of the most influential research topics in computational psychiatry to date, given the hypothesized relevance of these constructs as a means of capturing various forms of psychiatric dysfunction. Thus, a better understanding of the nature of the relationship between these constructs, coupled with a process of active refinement of the computational theory to achieve a much closer correspondence to the psychological constructs, is going to be critical for progress in this domain.