Determining the explanatory utility of computational reinforcement-learning theories of goal-directed and habitual control at behavioral and neural levels

点击翻译按钮获取中文摘要

Determining the explanatory utility of computational reinforcement-learning theories of goal-directed and habitual control at behavioral and neural levels PI: Dr. John P. O’Doherty Institution: California Institute of Technology PROJECT SUMMARY Accumulating evidence supports the existence of two distinct systems for guiding action-selection in the brain: a goal-directed system in which actions are selected with reference to the current incentive value of the associated goal or outcome, and a habitual system in which actions are selected reflexively, based solely on their history of past reinforcement. A computational account for these two systems has been formulated in terms of two distinct variants of computational reinforcement-learning (RL) theory: model-based (MB) vs model-free (MF) RL. Yet, empirical evidence in support of the proposed correspondence between the psychological (RDoC level) and computational RL accounts are sparse. Here we aim to comprehensively address whether the RDoC level constructs of goal-directed and habitual control can be effectively described by the computational framework of model-based and model-free RL in humans at both behavioral and neural levels. We plan to administer two distinct behavioral tasks designed to discriminate goal-directed from habitual control and MB from MF control to a large cohort of healthy participants (n=200) and an undifferentiated cohort of psychiatric patients (n=100). Our participants will perform these tasks while being scanned with fMRI, in addition to undergoing resting-state fMRI, and diffusion weighted imaging. We will also measure behavioral traits and states relevant to psychopathology in the same individuals. We will leverage individual differences across our behavioral, computational and neural measures in order to determine the extent to which the psychological constructs and computational accounts are best viewed as being one and the same, or whether by contrast they diverge in theoretically important ways. Should we detect clear differences between the psychological (RDoC) constructs and computational descriptions on any of the levels of analysis we utilize, this will motivate an iterative refinement of the computational framework to better approximate the psychological (RDoC) level constructs, to be accomplished in parallel to the experimental aims. The distinction between goals and habits and their proposed computational bases are arguably one of the most influential research topics in computational psychiatry to date, given the hypothesized relevance of these constructs as a means of capturing various forms of psychiatric dysfunction. Thus, a better understanding of the nature of the relationship between these constructs, coupled with a process of active refinement of the computational theory to achieve a much closer correspondence to the psychological constructs, is going to be critical for progress in this domain.