Evaluating Human–Robot Interaction Algorithms in Shared Autonomy via Quality Diversity Scenario Generation
Evaluating Human–Robot Interaction Algorithms in Shared Autonomy via Quality Diversity Scenario Generation
复制标题
DOI：
10.1145/3476412
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
Matthew C. Fontaine;S. Nikolaidis
中科院分区：
文献类型：
--
作者：
Matthew C. Fontaine;S. Nikolaidis
人类和机器人之间交互的规模和复杂性的增长凸显了对新计算方法来自动评估新算法和应用程序的需求。探索人类和机器人在模拟中交互的不同场景可以提高对机器人系统的理解，并避免现实世界中可能出现代价高昂的故障。我们将此问题表述为质量多样性（QD）问题，其目标是通过同时探索环境和人类行为来发现不同的故障场景。我们专注于共享自治领域，其中机器人试图推断人类操作员的目标，并采用 QD 算法 CMA-ME 和 MAP-Elites 为该领域中的两种已发布算法生成场景：通过事后优化和线性策略混合实现共享自治。一些生成的场景证实了之前的理论发现，而另一些则令人惊讶，并带来了对最先进实现的新理解。我们的实验表明，QD 算法 CMA-ME 和 MAP-Elites 在有效搜索场景空间方面优于基于蒙特卡罗模拟和优化的方法，突出了它们在人机交互中自动评估算法的前景。
The growth of scale and complexity of interactions between humans and robots highlights the need for new computational methods to automatically evaluate novel algorithms and applications. Exploring diverse scenarios of humans and robots interacting in simulation can improve understanding of the robotic system and avoid potentially costly failures in real-world settings. We formulate this problem as a quality diversity (QD) problem, of which the goal is to discover diverse failure scenarios by simultaneously exploring both environments and human actions. We focus on the shared autonomy domain, in which the robot attempts to infer the goal of a human operator, and adopt the QD algorithms CMA-ME and MAP-Elites to generate scenarios for two published algorithms in this domain: shared autonomy via hindsight optimization and linear policy blending. Some of the generated scenarios confirm previous theoretical findings, while others are surprising and bring about a new understanding of state-of-the-art implementations. Our experiments show that the QD algorithms CMA-ME and MAP-Elites outperform Monte-Carlo simulation and optimization-based methods in effectively searching the scenario space, highlighting their promise for automatic evaluation of algorithms in human–robot interaction.