Interactive Learning from Policy-Dependent Human Feedback
Interactive Learning from Policy-Dependent Human Feedback
复制标题
DOI：
--
复制
发表时间：
2017-01
期刊：
影响因子：
--
通讯作者：
J. MacGlashan;Mark K. Ho;R. Loftin;Bei Peng;Guan Wang;David L. Roberts;Matthew E. Taylor;M. Littman-M.
中科院分区：
文献类型：
--
作者：
J. MacGlashan;Mark K. Ho;R. Loftin;Bei Peng;Guan Wang;David L. Roberts;Matthew E. Taylor;M. Littman-M.
本文研究了人类教师使用正反馈和负反馈交流的交互式学习行为问题。许多以前的工作对这个问题的假设，人们提供反馈的决定，这是依赖于他们的行为教学和独立于学习者的当前政策。我们提出的实证结果表明，这种假设是错误的-无论是人类教练给一个积极的或消极的反馈的决定是由学习者的当前政策的影响。基于这一见解，我们介绍了收敛演员批评的人（COACH），一种算法，从政策依赖的反馈，收敛到局部最优的学习。最后，我们证明了COACH可以成功地学习物理机器人的多种行为。
This paper investigates the problem of interactively learning behaviors communicated by a human teacher using positive and negative feedback. Much previous work on this problem has made the assumption that people provide feedback for decisions that is dependent on the behavior they are teaching and is independent from the learner's current policy. We present empirical results that show this assumption to be false— whether human trainers give a positive or negative feedback for a decision is influenced by the learner's current policy. Based on this insight, we introduce Convergent Actor-Critic by Humans (COACH), an algorithm for learning from policy-dependent feedback that converges to a local optimum. Finally, we demonstrate that COACH can successfully learn multiple behaviors on a physical robot.