Learning to Assess Danger from Movies for Cooperative Escape Planning in Hazardous Environments
Learning to Assess Danger from Movies for Cooperative Escape Planning in Hazardous Environments
复制标题
DOI：
10.1109/iros47612.2022.9982279
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Vikram Shree;Sarah Allen;B. Asfora;Jacopo Banfi;Mark E. Campbell
中科院分区：
文献类型：
--
作者：
Vikram Shree;Sarah Allen;B. Asfora;Jacopo Banfi;Mark E. Campbell
在改善机器人感知和导航方面已经有了大量的工作，但它们在危险环境中的应用，如火灾或地震，仍处于初级阶段。我们在这里假设了两个关键挑战：第一，很难在现实世界中复制这样的场景，这对于培训和测试目的是必要的。其次，目前的系统不能充分利用在这种危险环境中可用的丰富的多模式数据。为了解决第一个挑战，我们建议利用电影和电视节目形式提供的海量可视内容，并开发一个可以表示现实世界中遇到的危险环境的数据集。用真实灾难图像的高级别危险等级对数据进行注释，并提供总结场景内容的相应关键字。针对第二个挑战，我们提出了一种用于人-机器人协同逃生场景的多通道危险估计流水线。我们的贝叶斯框架通过融合来自机器人摄像头传感器的信息和来自人类的语言输入来改进危险估计。此外，我们在评估模块中增加了一个风险感知规划器，帮助确定脱离危险环境的更安全的路径。通过广泛的模拟，我们展示了我们的多模式感知框架的优势，该框架转化为实实在在的好处，例如在协作的人-机器人任务中提高成功率。
There has been a plethora of work towards im-proving robot perception and navigation, yet their application in hazardous environments, like during a fire or an earthquake, is still at a nascent stage. We hypothesize two key challenges here: first, it is difficult to replicate such scenarios in the real world, which is necessary for training and testing purposes. Second, current systems are not fully able to take advantage of the rich multi-modal data available in such hazardous environments. To address the first challenge, we propose to harness the enormous amount of visual content available in the form of movies and TV shows, and develop a dataset that can represent hazardous environments encountered in the real world. The data is annotated with high-level danger ratings for realistic disaster images, and corresponding keywords are provided that summarize the content of the scene. In response to the second challenge, we propose a multi-modal danger estimation pipeline for collaborative human-robot escape scenarios. Our Bayesian framework improves danger estimation by fusing information from robot's camera sensor and language inputs from the human. Furthermore, we augment the estimation module with a risk-aware planner that helps in identifying safer paths out of the dangerous environment. Through extensive simulations, we exhibit the advantages of our multi-modal perception framework that gets translated into tangible benefits such as higher success rate in a collaborative human-robot mission.