SocNavBench: A Grounded Simulation Testing Framework for Evaluating Social Navigation
SocNavBench: A Grounded Simulation Testing Framework for Evaluating Social Navigation
复制标题
DOI：
10.1145/3476413
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Abhijat Biswas;Allan Wang;Gustavo Silvera;Aaron Steinfeld;H. Admoni
中科院分区：
文献类型：
--
作者：
Abhijat Biswas;Allan Wang;Gustavo Silvera;Aaron Steinfeld;H. Admoni
人机交互社区已经开发了许多方法，让机器人安全地与人类一起社交。然而，评估这些作品的实验程序通常是基于每种方法构建的。这种不同的评价使得很难在文献中比较这些方法的性能。为了弥合这一差距，我们引入了SocNavBench，一个用于评估社交导航算法的模拟框架。SocNavBench包括一个模拟器，具有照片般逼真的功能和基于真实世界行人数据的策划社交导航场景。我们还提供了一套指标来量化这些情况下的导航算法的性能的实现。总而言之，SocNavBench提供了一个测试框架，用于以一致和可解释的方式评估不同的社交导航方法。为了说明它的使用，我们演示了测试三个现有的社会导航方法和SocNavBench的基线方法，显示如何套指标有助于推断他们的性能权衡。我们的代码是开源的，允许社区添加新的场景和指标，以帮助发展SocNavBench，以反映我们对社交导航理解的进步。
The human-robot interaction community has developed many methods for robots to navigate safely and socially alongside humans. However, experimental procedures to evaluate these works are usually constructed on a per-method basis. Such disparate evaluations make it difficult to compare the performance of such methods across the literature. To bridge this gap, we introduce SocNavBench, a simulation framework for evaluating social navigation algorithms. SocNavBench comprises a simulator with photo-realistic capabilities and curated social navigation scenarios grounded in real-world pedestrian data. We also provide an implementation of a suite of metrics to quantify the performance of navigation algorithms on these scenarios. Altogether, SocNavBench provides a test framework for evaluating disparate social navigation methods in a consistent and interpretable manner. To illustrate its use, we demonstrate testing three existing social navigation methods and a baseline method on SocNavBench, showing how the suite of metrics helps infer their performance trade-offs. Our code is open-source, allowing the addition of new scenarios and metrics by the community to help evolve SocNavBench to reflect advancements in our understanding of social navigation.