Learn Task First or Learn Human Partner First: A Hierarchical Task Decomposition Method for Human-Robot Cooperation
Learn Task First or Learn Human Partner First: A Hierarchical Task Decomposition Method for Human-Robot Cooperation
复制标题
DOI：
10.1109/smc52423.2021.9659041
复制
发表时间：
2020-03
期刊：
影响因子：
--
通讯作者：
Lingfeng Tao;Michael Bowman;Jiucai Zhang;Xiaoli Zhang
中科院分区：
文献类型：
--
作者：
Lingfeng Tao;Michael Bowman;Jiucai Zhang;Xiaoli Zhang
将深度强化学习（DRL）应用于动态控制问题中的人机合作（HRC）是有前途的，但也具有挑战性，因为机器人需要学习受控系统的动力学和人类伙伴的动力学。在现有的研究中，由 DRL 驱动的机器人采用对环境和人类伙伴的耦合观察来同时学习两种动态。然而，这样的学习策略在学习效率和团队绩效方面受到限制。这项工作提出了一种具有分层奖励机制的新颖任务分解方法，使机器人能够独立于学习人类伙伴的行为来学习分层动态控制任务。该方法通过人体实验模拟环境中的分层控制任务进行了验证。我们的方法还提供了对 HRC 学习策略设计的深入了解。结果表明，机器人应该首先学习任务才能获得更高的团队绩效，而首先学习人类才能获得更高的学习效率。
Applying Deep Reinforcement Learning (DRL) to Human-Robot Cooperation (HRC) in dynamic control problems is promising yet challenging as the robot needs to learn the dynamics of the controlled system and dynamics of the human partner. In existing research, the robot powered by DRL adopts coupled observation of the environment and the human partner to learn both dynamics simultaneously. However, such a learning strategy is limited in terms of learning efficiency and team performance. This work proposes a novel task decomposition method with a hierarchical reward mechanism that enables the robot to learn the hierarchical dynamic control task separately from learning the human partner’s behavior. The method is validated with a hierarchical control task in a simulated environment with human subject experiments. Our method also provides insight into the design of the learning strategy for HRC. The results show that the robot should learn the task first to achieve higher team performance and learn the human first to achieve higher learning efficiency.