EAGER: Collaborative Research: Augmented 360 Video for Situation Awareness in Firefighting

火灾事件已造成消防员和平民的大量伤亡。控制中心的指挥官和在紧急地点工作的消防员之间沟通不畅，经常被认为是消防员行动死亡和损失报告的决定性因素。传统上，远程指挥官使用消防员头盔摄像头拍摄的视频来可视化紧急地点并领导行动。不幸的是，这些视频系统存在一个根本问题，即指挥官一次只能看到紧急现场的一个视图。这一限制限制了指挥官对情况的认识，并导致生产力和安全问题，如地点沟通错误和未能识别危险事件。该项目将360度视频和增强现实相结合，使远程指挥员能够在所有观看方向上实现对整个应急现场的360度态势感知，提高消防生产力和安全性。360度视频观看使各个应急响应社区在规划和培训方面受益。其他研究成果，包括开放的数据集和软件，通过出版物、演示文稿和网站广泛传播，为计算机科学和工程界做出贡献。还计划开展教育活动，包括本科生研究以及对K-12学生的消防安全培训，以加强该项目的影响。该项目将设计和开发增强的360视频技术，使远程指挥官能够在360度场景中切换视窗，并将机器检测到的感兴趣事件可视化。它将研究一个增强型360视频观看系统，通过以下步骤为消防员行动中的事件指挥提供全景态势感知：(1)针对指挥员在消防员视频中对态势感知的需求和要求，将进行访谈，以识别和分类消防员响应中的重要对象和事件及其技术偏好；(2)为了填补如何实现消防员视频中重要事件的自动机器检测的知识空白，将设计一种兴趣检测模型来检测目标对象和事件；为了向指挥官提供全景态势感知，将通过实地研究开发和评估用于远程事件指挥的增强型360视频观看系统。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Fire incidents have caused substantial injuries, illness, and death on both firefighters and civilians. Poor communications between commanders in the control center and firefighters working at emergency sites are frequently cited as the determining factor in fatality and loss reports of firefighter operations. Traditionally, remote commanders visualize emergency sites and lead the operation using videos captured by helmet cameras of firefighters. Unfortunately, these video systems suffer from a fundamental problem, i.e., commanders are only able to see a single view of the emergency site at a time. This limitation restricts the situation awareness of commanders and leads to productivity and safety issues such as miscommunication of locations and failure to identify dangerous events. This project combines 360-degree videos and augmented reality to enable remote commanders to achieve 360-degree situation awareness of the entire emergency site in all viewing directions and enhance firefighting productivity and safety. The 360-degree video viewing benefits various emergency response communities in planning and training. Other research outcomes, including open datasets and software, are widely disseminated through publications, presentations, and websites to contribute to the computer science and engineering communities. Educational activities including undergraduate research as well as fire safety training for K-12 students are also planned to enhance the impacts of this project. This project will design and develop augmented 360 video technology to enable remote commanders to switch viewports within a 360-degree scene and visualize machine-detected events of interest. It will investigate an augmented 360 video viewing system to enable panoramic situation awareness for incident command in firefighter operations through these steps: (1) to address commanders’ needs and requirements for situation awareness in firefighter videos, interviews will be performed to identify and categorize important objects and events in firefighter response and their technology preferences; (2) to fill in the knowledge gap of how to enable automatic machine detection of important events in firefighter videos, a view of interest detection model will be designed detects target objects and events; and (3) to provide panoramic situation awareness to commanders, an augmented 360 video viewing system for remote incident command will be developed and evaluated through a field study.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.