Reinforcement learning signals predict future decisions
Reinforcement learning signals predict future decisions
复制标题
DOI：
10.1523/jneurosci.4421-06.2007
复制
发表时间：
2007-01-10
影响因子：
5.3
通讯作者：
Ranganath, Charan
中科院分区：
文献类型：
--
作者：
Cohen, Michael X.;Ranganath, Charan
在竞争激烈的世界中，最佳行为需要根据最近的结果灵活地调整决策策略。在本研究中，我们测试了一个假设，即这种灵活性是通过强化学习过程出现的，在这个过程中，奖励预测误差被动态地用来调整决策选项的表示。我们记录了事件相关脑电位（ERPs），同时受试者与计算机对手进行战略经济游戏，以评估神经对结果的反应如何与随后的决策相关。对ERP数据的分析集中在反馈相关负性（FRN）上，这是一种被认为反映神经预测错误信号的结果锁定电位。与计算强化学习模型的预测一致，我们发现，输给计算机对手后的ERP幅度预测了受试者是否会在随后的试验中改变决策行为。此外，FRN的决策结果是不成比例地大于运动皮层对侧的反应手，用于作出决定。这些发现提供了新的证据，证明人类参与强化学习过程来调整竞争决策选项的表示。
Optimal behavior in a competitive world requires the flexibility to adapt decision strategies based on recent outcomes. In the present study, we tested the hypothesis that this flexibility emerges through a reinforcement learning process, in which reward prediction errors are used dynamically to adjust representations of decision options. We recorded event-related brain potentials (ERPs) while subjects played a strategic economic game against a computer opponent to evaluate how neural responses to outcomes related to subsequent decision-making. Analyses of ERP data focused on the feedback-related negativity (FRN), an outcome-locked potential thought to reflect a neural prediction error signal. Consistent with predictions of a computational reinforcement learning model, we found that the magnitude of ERPs after losing to the computer opponent predicted whether subjects would change decision behavior on the subsequent trial. Furthermore, FRNs to decision outcomes were disproportionately larger over the motor cortex contralateral to the response hand that was used to make the decision. These findings provide novel evidence that humans engage a reinforcement learning process to adjust representations of competing decision options.