RAPID/Collaborative Research: Datasets for Uncrewed Aerial System (UAS) and Remote Responder Performance from Hurricane Ian

RAPID/协作研究：飓风伊恩无人飞行系统 (UAS) 和远程响应器性能的数据集

• 批准号: 2306453
• 负责人: Robin Murphy
• 金额: $ 14.48万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-02-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2306453&HistoricalAwards=false
• 关键词: RAPID; Collaborative; Research; Datasets; Uncrewed

This Grants for Rapid Response Research (RAPID) project will curate, supplement, and analyze data collected over a period of intensive uncrewed aerial system (UAS) operations, carried out as part of the State of Florida’s response to Hurricane Ian. From September 27, 2022, just before Hurricane Ian made landfall, and continually for the next nine days, teams from Florida State University and Texas A&M University helped coordinate 24 UAS pilots flying 16 different models of fixed-wing and rotorcraft UAS over Charlotte, Lee, and Hardee counties. These missions obtained aerial imagery to survey wind and flood damage, direct ground response, support strategic planning and resource allocation, monitor threats to public safety, and provide documentation for subsequent emergency relief funding. Under this award, the research team will curate 55,000 images and videos collected during the disaster, comprising over 750 gigabytes of data, and supporting material such as flight schedules and log files. The curated data, and derived products such as aerial maps and edited video, will be made publicly available for open-source use. The project will analyze the mission logs and data products to assess pilot performance over time, and will document variables potentially influencing pilot performance, including pilot skill, prior training and experience, operations tempo, and fatiguing conditions, supplemented by individual and collective interviews with the UAS pilots. The image dataset and derived products will help the computer vision/machine learning (CV/ML) community design better algorithms for identifying threats to public safety, damaged structures, and people in distress. The pilot performance dataset will be made available to the research community, to characterize human-robot performance, formulate best practices, and to understand deviation in behaviors and sources of mission error. The resulting insights into proper matching of vehicles, pilots, missions, and operational parameters will increase the ability of UAS platforms and pilots to save lives and accelerate economic recovery after a disaster. The datasets can help the domestic UAS industry improve products for response to a broad class of natural disasters, including wildfires and flooding, and for use in extreme environments, such as in oil and gas exploration and extraction and for in nuclear reactors and nuclear waste sites. The project will support the creation of better workflow procedures to reduce human error, increasing trust in the technology by UAS operators and other first responders, and facilitating adoption of UAS for emergency response. The project will broaden participation in science, with three women out of the four co-PIs, and will engage STEM students to help annotate the UAS imagery. This project will curate vehicle and pilot data from the deployment of uncrewed aerial systems (UAS) during Hurricane Ian by Florida State University and Texas A&M University for the robotics, computer vision/machine learning (CV/ML), human-robot interaction, and geospatial land-use communities. The project has the following three objectives: 1) Curate the data (imagery, log files, flight schedules, etc.) and data products (images, video, orthomosaic maps, digital surface maps) collected during the disaster and make available for open-source use; 2) Interview the UAS pilots individually and collectively in order to capture human-robot performance, best practices, deviation in behaviors, and sources of error; and 3) Analyze the mission logs and data products for performance (quality or completeness) and document the quality over time by pilots, prior training, and frequency of flying the missions in normative conditions, the operations tempo, and fatiguing conditions. From a robotics perspective, it will contribute to the emerging model of how multiple agents may be used during disasters, and the consequences for design, performance specifications, the role of artificial intelligence, and wireless communications. Such a model can greatly increase the competitiveness of the domestic drone industry, as well as motivate novel directions in swarm research. Research stemming from this project will generate guidelines for data collection in future disasters, setting the stage for advances in engineering and computing for disasters. It will increase the availability of training data for CV/ML and serve as a testbed for transfer of learned features from other disasters; both of which could lead to fundamental advances in machine learning. From a human-factors perspective, it will generate a new methodology for creating human-robot datasets that combine on-site direct data (with no experimenters in the field) with post-event data. This methodology would overcome current barriers in conducting empirical investigations into scientific questions on extreme work environments because of the prohibition on embedded experimenters. This methodology is expected to transfer to other extreme work environments, such as nuclear, space, oil and gas industry, and the military. The human-robot data itself could lead to major findings in human error and workforce training. From a geospatial perspective, the data can help establish the impact of rising sea levels, the built environment, and prior storm surge and flooding mitigations. Overall, the project will benefit society by increasing the ability to save lives and accelerate economic recovery after a disaster with UAS and is expected to create findings and methods that will generalize to new technologies for extreme environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该快速反应研究拨款（Rapid）项目将整理、补充和分析在一段时间内密集的无人机系统（UAS）操作中收集的数据，作为佛罗里达州应对飓风“伊恩”的一部分。从2022年9月27日，就在飓风伊恩登陆之前，在接下来的9天里，来自佛罗里达州立大学和德克萨斯农机大学的团队帮助协调24名无人机飞行员，驾驶16种不同型号的固定翼和旋翼无人机在夏洛特、李和哈迪县上空飞行。这些特派团获得的航空图像用于调查风灾灾害、指导地面反应、支持战略规划和资源分配、监测对公共安全的威胁以及为随后的紧急救济供资提供文件。根据该合同，研究团队将整理灾难期间收集的55,000张图像和视频，包括超过750 gb的数据，以及航班时刻表和日志文件等支持材料。经过整理的数据和衍生产品，如航空地图和编辑过的视频，将公开供开源使用。该项目将分析任务日志和数据产品，以评估飞行员在一段时间内的表现，并将记录可能影响飞行员表现的变量，包括飞行员技能、先前的训练和经验、操作节奏和疲劳条件，并辅以对无人机系统飞行员的个人和集体访谈。图像数据集和衍生产品将帮助计算机视觉/机器学习（CV/ML）社区设计更好的算法，以识别对公共安全、受损结构和遇险人员的威胁。试点性能数据集将提供给研究界，以表征人机性能，制定最佳实践，并了解行为偏差和任务错误的来源。由此产生的对车辆、飞行员、任务和操作参数的适当匹配的见解将提高UAS平台和飞行员在灾难后拯救生命和加速经济恢复的能力。这些数据集可以帮助国内无人机行业改进产品，以应对各种自然灾害，包括野火和洪水，以及在极端环境下的使用，如石油和天然气勘探和开采，以及核反应堆和核废料场。该项目将支持创建更好的工作流程，以减少人为错误，增加UAS操作员和其他第一响应者对技术的信任，并促进采用UAS进行应急响应。该项目将扩大对科学的参与，四名共同项目负责人中有三名是女性，并将邀请STEM学生帮助注释无人机图像。该项目将整理佛罗里达州立大学和德克萨斯农工大学在飓风伊恩期间部署的无人驾驶航空系统（UAS）的车辆和试点数据，用于机器人、计算机视觉/机器学习（CV/ML）、人机交互和地理空间土地利用社区。该项目有以下三个目标：1)整理灾难期间收集的数据（图像、日志文件、航班时刻表等）和数据产品（图像、视频、正射影图、数字地面图），并提供开源使用；2)单独和集体采访无人机飞行员，以捕捉人机性能、最佳实践、行为偏差和错误来源；3)分析任务日志和数据产品的性能（质量或完整性），并记录飞行员、先前训练、在规范条件下飞行任务的频率、操作节奏和疲劳条件下的质量随时间变化。从机器人的角度来看，它将有助于在灾难期间如何使用多个代理的新兴模型，以及对设计、性能规范、人工智能角色和无线通信的影响。这样的模型可以大大提高国内无人机产业的竞争力，也可以激发蜂群研究的新方向。该项目的研究将为未来灾害中的数据收集制定指导方针，为灾害工程和计算的进步奠定基础。它将增加CV/ML训练数据的可用性，并作为从其他灾难中迁移学习特征的测试平台；这两者都可能导致机器学习的根本性进步。从人为因素的角度来看，它将产生一种新的方法来创建人机数据集，将现场直接数据（没有现场实验人员）与事后数据相结合。由于禁止嵌入实验者，这种方法将克服目前在对极端工作环境的科学问题进行实证调查方面的障碍。这种方法有望转移到其他极端工作环境中，如核能、太空、石油和天然气工业以及军事。人机数据本身可能会导致在人为错误和劳动力培训方面的重大发现。从地理空间的角度来看，这些数据可以帮助确定海平面上升、建筑环境以及先前风暴潮和洪水缓解措施的影响。总的来说，该项目将通过提高UAS在灾难发生后拯救生命和加速经济复苏的能力来造福社会，并有望创造出适用于极端环境的新技术的发现和方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Wireless Network Demands of Data Products from Small Uncrewed Aerial Systems at Hurricane Ian

伊恩飓风期间小型无人航空系统数据产品的无线网络需求

• DOI: 10.1109/iros55552.2023.10342413
• 发表时间: 2023
• 期刊: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Manzini, Thomas;Murphy, Robin;Merrick, David;Adams, Justin
• 通讯作者: Adams, Justin

Harnessing AI and robotics in humanitarian assistance and disaster response

在人道主义援助和灾难应对中利用人工智能和机器人技术

• DOI: 10.1126/scirobotics.adj2767
• 发表时间: 2023
• 期刊: Science Robotics
• 影响因子: 25
• 作者: Manzini, Thomas;Murphy, Robin R.;Heim, Eric;Robinson, Caleb;Zarrella, Guido;Gupta, Ritwik
• 通讯作者: Gupta, Ritwik