Credit assignment in hierarchical option transfer
Credit assignment in hierarchical option transfer
复制标题
DOI：
--
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Jing-Jing Li-Jing;Liyu Xia;Flora Dong;Anne G. E. Collins
中科院分区：
文献类型：
--
作者：
Jing-Jing Li-Jing;Liyu Xia;Flora Dong;Anne G. E. Collins
人类具有在学习过程中有效地组织过去的知识以实现快速概括的非凡能力。夏和柯林斯(2021)在一项层次分明的顺序决策任务中评估了这种能力，参与者可以在时间和状态抽象的多个层面上建立“选项”(策略“块”)。一种名为期权模型的量化模型捕捉到了在人类参与者身上观察到的转移效应，这表明人类创造和组成了分层的选项，并使用它们来探索新的背景。然而，在新的背景下学习如何归因于新的和旧的选择(即学分分配问题)并没有被很好地理解。在一个有新的突发事件的新环境中，参与者可以重新组合以前学习的期权的某些方面，他们是可靠地创建新的期权还是覆盖现有的期权？信用分配取决于新选项与旧选项的相似程度吗？在我们的实验中，两组参与者(n=124和n=104)学习了分层结构的选项，在新的选项环境中经历了不同程度的负迁移，并随后在之前学习的选项上进行了测试。行为分析表明，旧期权在没有干扰的情况下被成功重用，新期权被适当地创建和计入信用。这一学分分配并不取决于新选项与旧选项的相似程度，显示出人类分层学习的极大灵活性和精确度。这些行为结果被选项模型捕获，为人类的选项学习和迁移提供了进一步的证据。
Humans have the exceptional ability to efficiently structure past knowledge during learning to enable fast generalization. Xia and Collins (2021) evaluated this ability in a hierarchically structured, sequential decision-making task, where participants could build “options” (strategy “chunks”) at multiple levels of temporal and state abstraction. A quantitative model, the Option Model, captured the transfer effects observed in human participants, suggesting that humans create and compose hierarchical options and use them to explore novel contexts. However, it is not well understood how learning in a new context is attributed to new and old options (i.e., the credit assignment problem). In a new context with new contingencies, where participants can recompose some aspects of previously learned options, do they reliably create new options or overwrite existing ones? Does the credit assignment depend on how similar the new option is to an old one? In our experiment, two groups of participants (n=124 and n=104) learned hierarchically structured options, experienced different amounts of negative transfer in a new option context, and were subsequently tested on the previously learned options. Behavioral analysis showed that old options were successfully reused without interference, and new options were appropriately created and credited. This credit assignment did not depend on how similar the new option was to the old option, showing great flexibility and precision in human hierarchical learning. These behavioral results were captured by the Option Model, providing further evidence for option learning and transfer in humans.