What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience.
What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience.
复制标题
DOI：
10.1016/j.cobeha.2021.06.004
复制
发表时间：
2021-10
影响因子：
5
通讯作者：
Collins AGE
中科院分区：
文献类型：
--
作者：
Eckstein MK;Wilbrecht L;Collins AGE
强化学习（RL）是一个对机器学习、神经科学和认知科学等领域都非常宝贵的概念。然而，RL所需要的不同领域，导致解释和翻译结果时的困难。在列出这些差异之后，本文将重点放在认知（神经）科学上，讨论我们作为一个领域如何过度解释RL建模结果。我们经常假设-隐含地-建模结果在任务，模型和参与者群体之间通用，尽管这种假设有负面的经验证据。我们还经常假设参数测量特定的，独特的（神经）认知过程，我们称之为可解释性的概念，当证据表明它们在研究和任务中捕获不同的功能时。我们的结论是，未来的计算研究需要更多地关注使用RL模型时的隐式假设，并建议更系统地了解上下文因素将有助于解决问题，提高RL解释大脑和行为的能力。
Reinforcement learning (RL) is a concept that has been invaluable to fields including machine learning, neuroscience, and cognitive science. However, what RL entails differs between fields, leading to difficulties when interpreting and translating findings. After laying out these differences, this paper focuses on cognitive (neuro)science to discuss how we as a field might over-interpret RL modeling results. We too often assume—implicitly—that modeling results generalize between tasks, models, and participant populations, despite negative empirical evidence for this assumption. We also often assume that parameters measure specific, unique (neuro)cognitive processes, a concept we call interpretability, when evidence suggests that they capture different functions across studies and tasks. We conclude that future computational research needs to pay increased attention to implicit assumptions when using RL models, and suggest that a more systematic understanding of contextual factors will help address issues and improve the ability of RL to explain brain and behavior.
登录
查看更多内容
DOI：
10.1073/pnas.1720963115
发表时间：
2018-03-06
影响因子：
11.1
作者：
Collins, Anne G. E.;Frank, Michael J.
通讯作者：
Frank, Michael J.
影响因子：
5.4
作者：
Collins, Anne G. E.;Frank, Michael J.
通讯作者：
Frank, Michael J.
影响因子：
16.2
作者：
Daw ND;Gershman SJ;Seymour B;Dayan P;Dolan RJ
通讯作者：
Dolan RJ
影响因子：
3.4
作者：
Blohm, Gunnar;Kording, Konrad P.;Schrater, Paul R.
通讯作者：
Schrater, Paul R.
影响因子：
7.7
作者：
Bakkour, Akram;Palombo, Daniela J.;Shohamy, Daphna
通讯作者：
Shohamy, Daphna