EAGER: Collaborative Research: Augmented 360 Video for Situation Awareness in Firefighting

火灾事件对消防员和平民造成了严重的伤害、疾病和死亡。控制中心的指挥官和在紧急地点工作的消防员之间沟通不畅，经常被认为是消防员行动中死亡和损失报告的决定性因素。传统上，远程指挥员利用消防员头盔摄像机拍摄的视频可视化紧急地点并领导行动。不幸的是，这些视频系统存在一个根本问题，即指挥官一次只能看到紧急地点的单一视图。这种限制限制了指挥官的情况意识，并导致生产力和安全问题，例如位置的错误沟通和未能识别危险事件。该项目结合了360度视频和增强现实，使远程指挥官能够在所有观看方向上实现对整个紧急情况现场的360度情况感知，并提高消防生产力和安全性。360度视频观看使各种应急响应社区在规划和培训方面受益。其他研究成果，包括开放数据集和软件，通过出版物、演讲和网站广泛传播，为计算机科学和工程社区做出贡献。教育活动，包括本科研究和消防安全培训为K-12学生也计划提高这个项目的影响。该项目将设计和开发增强360视频技术，使远程指挥官能够在360度场景中切换视口，并可视化机器检测到的感兴趣的事件。它将研究一种增强360视频观看系统，通过以下步骤为消防员行动中的事件指挥提供全景态势感知：(1)为了满足指挥官对消防员视频中态势感知的需求和要求，将进行访谈，以识别和分类消防员响应中的重要对象和事件及其技术偏好；(2)为填补如何实现消防员视频中重要事件的机器自动检测的知识空白，设计兴趣视图检测模型，检测目标物体和事件；(3)为向指挥官提供全景态势感知，将开发用于远程事件指挥的增强360视频观看系统，并通过实地研究进行评估。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Fire incidents have caused substantial injuries, illness, and death on both firefighters and civilians. Poor communications between commanders in the control center and firefighters working at emergency sites are frequently cited as the determining factor in fatality and loss reports of firefighter operations. Traditionally, remote commanders visualize emergency sites and lead the operation using videos captured by helmet cameras of firefighters. Unfortunately, these video systems suffer from a fundamental problem, i.e., commanders are only able to see a single view of the emergency site at a time. This limitation restricts the situation awareness of commanders and leads to productivity and safety issues such as miscommunication of locations and failure to identify dangerous events. This project combines 360-degree videos and augmented reality to enable remote commanders to achieve 360-degree situation awareness of the entire emergency site in all viewing directions and enhance firefighting productivity and safety. The 360-degree video viewing benefits various emergency response communities in planning and training. Other research outcomes, including open datasets and software, are widely disseminated through publications, presentations, and websites to contribute to the computer science and engineering communities. Educational activities including undergraduate research as well as fire safety training for K-12 students are also planned to enhance the impacts of this project. This project will design and develop augmented 360 video technology to enable remote commanders to switch viewports within a 360-degree scene and visualize machine-detected events of interest. It will investigate an augmented 360 video viewing system to enable panoramic situation awareness for incident command in firefighter operations through these steps: (1) to address commanders’ needs and requirements for situation awareness in firefighter videos, interviews will be performed to identify and categorize important objects and events in firefighter response and their technology preferences; (2) to fill in the knowledge gap of how to enable automatic machine detection of important events in firefighter videos, a view of interest detection model will be designed detects target objects and events; and (3) to provide panoramic situation awareness to commanders, an augmented 360 video viewing system for remote incident command will be developed and evaluated through a field study.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.