Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package.
Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package.
复制标题
使用 hBayesDM 软件包揭示强化学习和决策的神经计算机制。
DOI：
10.1162/cpsy_a_00002
复制
发表时间：
2017-10
期刊：
影响因子：
--
通讯作者：
Zhang L
中科院分区：
文献类型：
--
作者：
Ahn WY;Haines N;Zhang L
强化学习和决策（RLDM）提供了一个定量框架和计算理论，我们可以利用它们将精神疾病分解为神经认知功能的基本维度。 RLDM 提供了一种评估和潜在诊断精神病患者的新方法，临床研究人员对 RLDM 和计算精神病学的热情日益高涨。这样的框架还可以深入了解特定 RLDM 过程的大脑基质，例如对功能性磁共振成像 (fMRI) 或脑电图 (EEG) 数据进行基于模型的分析。然而，研究人员经常发现这种方法技术性太强，很难将其用于他们的研究。因此，仍然迫切需要开发一种用户友好的工具来广泛传播计算精神病学方法。我们引入了一个名为 hBayesDM（决策任务的分层贝叶斯建模）的 R 包，它提供了一系列 RLDM 任务和社交交换游戏的计算建模。 hBayesDM 包提供最先进的分层贝叶斯建模，其中个体参数和组参数（即后验分布）以相互约束的方式同时估计。同时，该软件包非常用户友好：用户可以执行计算建模、输出可视化和贝叶斯模型比较，每项操作都只需一行代码。用户还可以提取基于模型的 fMRI/EEG 所需的逐次试验潜在变量（例如预测误差）。通过 hBayesDM 软件包，我们预计任何具有最少编程知识的人都可以利用尖端的计算建模方法来研究多个决策系统（例如目标导向、习惯和巴甫洛夫）系统的底层过程和相互作用。通过这种方式，我们期望 hBayesDM 软件包将有助于先进建模方法的传播，并使广泛的研究人员能够轻松地在不同人群中进行计算精神病学研究。
Reinforcement learning and decision-making (RLDM) provide a quantitative framework and computational theories with which we can disentangle psychiatric conditions into the basic dimensions of neurocognitive functioning. RLDM offer a novel approach to assessing and potentially diagnosing psychiatric patients, and there is growing enthusiasm for both RLDM and computational psychiatry among clinical researchers. Such a framework can also provide insights into the brain substrates of particular RLDM processes, as exemplified by model-based analysis of data from functional magnetic resonance imaging (fMRI) or electroencephalography (EEG). However, researchers often find the approach too technical and have difficulty adopting it for their research. Thus, a critical need remains to develop a user-friendly tool for the wide dissemination of computational psychiatric methods. We introduce an R package called hBayesDM (hierarchical Bayesian modeling of Decision-Making tasks), which offers computational modeling of an array of RLDM tasks and social exchange games. The hBayesDM package offers state-of-the-art hierarchical Bayesian modeling, in which both individual and group parameters (i.e., posterior distributions) are estimated simultaneously in a mutually constraining fashion. At the same time, the package is extremely user-friendly: users can perform computational modeling, output visualization, and Bayesian model comparisons, each with a single line of coding. Users can also extract the trial-by-trial latent variables (e.g., prediction errors) required for model-based fMRI/EEG. With the hBayesDM package, we anticipate that anyone with minimal knowledge of programming can take advantage of cutting-edge computational-modeling approaches to investigate the underlying processes of and interactions between multiple decision-making (e.g., goal-directed, habitual, and Pavlovian) systems. In this way, we expect that the hBayesDM package will contribute to the dissemination of advanced modeling approaches and enable a wide range of researchers to easily perform computational psychiatric research within different populations.