Exploiting Natural Language for Efficient Risk-Aware Multi-Robot SaR Planning
Exploiting Natural Language for Efficient Risk-Aware Multi-Robot SaR Planning
复制标题
DOI：
10.1109/lra.2021.3062798
复制
发表时间：
2021-04
影响因子：
5.2
通讯作者：
Vikram Shree;B. Asfora;Rachel Zheng;Samantha Hong;Jacopo Banfi;M. Campbell
中科院分区：
文献类型：
--
作者：
Vikram Shree;B. Asfora;Rachel Zheng;Samantha Hong;Jacopo Banfi;M. Campbell
发展对场景的高级理解的能力，如感知危险级别，在规划多机器人搜索和救援(SAR)任务时可能被证明是有价值的。在这项工作中，我们建议独特地利用任务总司令的自然语言描述和机器人捕获的图像数据来估计场景危险。在给定描述和图像的情况下，使用最先进的深度神经网络来评估相应的相似性分数，然后将其转换为危险级别的概率分布。由于常用的视觉语言数据集不能很好地代表SAR任务，我们从真实灾难场景的合成图像中收集了一个大规模的图像描述数据集，并使用它来训练我们的机器学习模型。然后将多机器人有效搜索路径规划(MESPP)问题的一种具有风险意识的变体表示为使用危险估计，以便在规划搜索者的路径时考虑环境中的高风险位置。该问题通过基于混合整数线性规划的分布式方法来解决。我们的实验表明，我们的框架允许计划更安全但非常成功的搜索任务，遵守搜索任务的两个最重要的方面：确保搜索者和受害者的安全。
The ability to develop a high-level understanding of a scene, such as perceiving danger levels, can prove valuable in planning multi-robot search and rescue (SaR) missions. In this work, we propose to uniquely leverage natural language descriptions from the mission commander in chief and image data captured by robots to estimate scene danger. Given a description and an image, a state-of-the-art deep neural network is used to assess a corresponding similarity score, which is then converted into a probabilistic distribution of danger levels. Because commonly used visio-linguistic datasets do not represent SaR missions well, we collect a large-scale image-description dataset from synthetic images taken from realistic disaster scenes and use it to train our machine learning model. A risk-aware variant of the Multi-robot Efficient Search Path Planning (MESPP) problem is then formulated to use the danger estimates in order to account for high-risk locations in the environment when planning the searchers’ paths. The problem is solved via a distributed approach based on Mixed-Integer Linear Programming. Our experiments demonstrate that our framework allows to plan safer yet highly successful search missions, abiding to the two most important aspects of SaR missions: to ensure both searchers’ and victim safety.