Computional interactions between model-free and model-based decision making in the human brain

适应性行为的一个基本原则是，人类和动物寻求最大化奖励和最小化惩罚。已经证实，大脑采用两种互补的计算策略来解决这个任务：一种是无模型重复先前成功的行为，另一种是更苛刻的基于模型的方法来评估哪些未来行为将导致最有价值的结果。与这些策略相一致的是，决策过程中的神经影像学研究揭示了与大脑中广泛训练后的计划和选择有关的不同价值信号的证据。然而，这两个系统在学习和选择中是孤立地起作用还是相互作用，还有待观察。此外，决定哪个系统控制行为的因素仍然不清楚。在这个建议中，我们概述了一系列全面的调查，在人类中研究无模型和基于模型的选择系统之间的相互作用，并揭示在大脑中的信号对应于底层的计算过程。具体来说，我们解决了系统之间的信息传输问题，例如通过规划系统或基于模型的教学信号来访问学习的缓存值以指示缓存值。然后，我们将研究决定无模型或基于模型的系统是否影响选择以及影响程度的因素和神经机制。特别是，我们将研究一个假定的作用，相对稳定性在两个控制器的预测行为控制，和工作记忆的性能在两个系统中的作用。最后，我们将通过比较抑郁症受试者和健康对照者之间的无模型和基于模型的选择表现以及潜在的神经系统来探索决策在精神病理学中的作用。我们希望这些发现将大大增强我们对人类大脑如何实现决策基础计算的了解，也可能帮助我们理解和描述精神病患者的决策缺陷。

A basic tenet of adaptive behavior is that humans and animals seek to maximize reward and minimize punishment. It has been conjectured that the brain employs two complementary computational strategies to solve this task: a model-free repetition of previously successful behavior, and a more demanding model‐based approach to assess which future actions will lead to the most highly valued outcomes. Consistent with these strategies, neuroimaging studies during decision-making have revealed evidence for distinct value signals pertaining to planning and choice after extensive training in the brain. However, it remains to be seen whether both systems act in isolation or interact in learning and choice. Furthermore, the factors governing which system controls behavior are still unclear. In this proposal we outline a comprehensive series of investigations in humans to study interactions between model-free and model-based choice systems, and to uncover signals in the brain corresponding to the underlying computational processes. Specifically, we address the question of information transfer between systems, such as in accessing learnt cached values by a planning system or a model‐based teaching signal to instruct cached values. We will then investigate the factors and neural mechanisms that determine whether model‐free or model-based systems influence choice and to what degree. Particularly, we will study a putative role of relative certainties in both controllers’ predictions for behavioral control, and the role of working memory performance in both systems. Finally, we will explore the role of decision making in psychopathology by comparing model-free and model‐based choice performance and underlying neural systems between depressed subjects and healthy controls. We expect that the findings will greatly enhance our knowledge about how the human brain implements the computations underlying decision-making and may also help us understand and characterize decision-making deficits in psychiatric patients.