Cheap but Clever: Human Active Learning in a Bandit Setting
Cheap but Clever: Human Active Learning in a Bandit Setting
复制标题
廉价但聪明：强盗环境中的人类主动学习
DOI：
--
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
Angela J. Yu
中科院分区：
文献类型：
--
作者：
Shunan Zhang;Angela J. Yu
人们如何在不完全了解的环境中通过反复尝试和嘈杂的结果来实现长期目标，这是认知科学中的一个重要问题。有两个相互关联的问题：人类如何表达信息，包括已经学到的和仍然可以学到的信息，以及他们如何选择行动，特别是他们如何协调探索和开发之间的紧张关系。在这项工作中，我们检查了多臂强盗设置中的人类行为数据，在该环境中，受试者从四个“手臂”中选择一个来拉动每个试验，并收到一个二元结果(赢/输)。我们实现了贝叶斯最优策略和各种不同的启发式策略，这些策略在信息表示和决策策略的复杂性上有所不同。我们发现，知识梯度算法结合了精确的贝叶斯学习和最大化短期回报收益和长期知识收益组合的决策策略，在所有考虑的模型中捕获受试者的逐次尝试选择最好；在所有启发式策略中，它也提供了计算量最大的最优策略的最佳逼近。
How people achieve long-term goals in an imperfectly known environment, via repeated tries and noisy outcomes, is an important problem in cognitive science. There are two interrelated questions: how humans represent information, both what has been learned and what can still be learned, and how they choose actions, in particular how they negotiate the tension between exploration and exploitation. In this work, we examine human behavioral data in a multi-armed bandit setting, in which the subject choose one of four “arms” to pull on each trial and receives a binary outcome (win/lose). We implement both the Bayes-optimal policy, which maximizes the expected cumulative reward in this finite-horizon bandit environment, as well as a variety of heuristic policies that vary in their complexity of information representation and decision policy. We find that the knowledge gradient algorithm, which combines exact Bayesian learning with a decision policy that maximizes a combination of immediate reward gain and longterm knowledge gain, captures subjects’ trial-by-trial choice best among all the models considered; it also provides the best approximation to the computationally intense optimal policy among all the heuristic policies.
DOI：
--
发表时间：
2008-12
期刊：
Advances in neural information processing systems
影响因子：
--
作者：
Angela J. Yu;J. Cohen
通讯作者：
Angela J. Yu;J. Cohen