Two sides of the same coin: Beneficial and detrimental consequences of range adaptation in human reinforcement learning
Two sides of the same coin: Beneficial and detrimental consequences of range adaptation in human reinforcement learning
复制标题
DOI：
10.1126/sciadv.abe0340
复制
发表时间：
2021-03-01
期刊：
影响因子：
13.6
通讯作者：
Palminteri, Stefano
中科院分区：
文献类型：
--
作者：
Bavard, Sophie;Rustichini, Aldo;Palminteri, Stefano
有证据表明，经济价值是根据可供选择的范围重新调整的。虽然局部自适应，但范围自适应已被证明会导致次优选择，特别是在强化学习（RL）情况下，当选项从原始上下文外推到新上下文时。范围自适应可以被视为旨在增加信噪比的自适应编码过程的结果。然而，这一假设导致了一个违反直觉的预测：降低任务难度会增加范围适应，从而增加外推误差。在这里，我们测试了范围适应和性能之间的矛盾关系，在一个大样本的参与者执行的RL任务的变体，在那里我们操纵任务难度。结果证实，范围适应引起系统外推误差，并在降低任务难度时更强。最后，我们提出了一个范围自适应模型，并表明它能够简约地捕捉所有的行为结果。
Evidence suggests that economic values are rescaled as a function of the range of the available options. Although locally adaptive, range adaptation has been shown to lead to suboptimal choices, particularly notable in reinforcement learning (RL) situations when options are extrapolated from their original context to a new one. Range adaptation can be seen as the result of an adaptive coding process aiming at increasing the signal-to-noise ratio. However, this hypothesis leads to a counterintuitive prediction: Decreasing task difficulty should increase range adaptation and, consequently, extrapolation errors. Here, we tested the paradoxical relation between range adaptation and performance in a large sample of participants performing variants of an RL task, where we manipulated task difficulty. Results confirmed that range adaptation induces systematic extrapolation errors and is stronger when decreasing task difficulty. Last, we propose a range-adapting model and show that it is able to parsimoniously capture all the behavioral results.