RAPID/Collaborative Research: Datasets for Uncrewed Aerial System (UAS) and Remote Responder Performance from Hurricane Ian

这一快速反应研究拨款(RAPID)项目将对在密集的无人驾驶空中系统(UAS)操作期间收集的数据进行管理、补充和分析，作为佛罗里达州应对飓风Ian的一部分。从2022年9月27日，就在飓风伊恩登陆之前，在接下来的9天里，来自佛罗里达州立大学和德克萨斯农工大学的团队帮助协调了24名无人机飞行员，驾驶着16种不同型号的固定翼和旋翼无人机在夏洛特县、利县和哈迪县上空飞行。这些特派团获得了航空图像，以调查风灾和洪灾损失，指导地面反应，支持战略规划和资源分配，监测对公共安全的威胁，并为随后的紧急救济资金提供文件。根据这一奖项，研究团队将整理灾难期间收集的55,000张图像和视频，其中包括超过750 GB的数据，以及航班时刻表和日志文件等辅助材料。经过整理的数据及其衍生产品，如航空地图和编辑后的视频，将公开供开源使用。该项目将分析任务日志和数据产品，以评估飞行员随时间的表现，并将记录可能影响飞行员表现的变量，包括飞行员技能、先前的培训和经验、操作节奏和疲劳状况，并辅之以对无人机飞行员的个人和集体采访。图像数据集和衍生产品将帮助计算机视觉/机器学习(CV/ML)社区设计更好的算法，以识别对公共安全、受损结构和遇险人员的威胁。飞行员性能数据集将提供给研究界，以确定人-机器人性能的特征，制定最佳做法，并了解行为偏差和任务错误的来源。由此产生的对车辆、飞行员、任务和操作参数的适当匹配的洞察将增加UAS平台和飞行员在灾难后拯救生命和加速经济恢复的能力。这些数据集可以帮助国内UAS行业改进产品，以应对包括野火和洪水在内的广泛类别的自然灾害，并用于极端环境，如石油和天然气勘探和开采，以及核反应堆和核废料场地。该项目将支持创建更好的工作流程程序，以减少人为错误，增加无人机操作员和其他急救人员对该技术的信任，并促进采用无人机进行紧急响应。该项目将扩大对科学的参与，四名共同PIs中有三名女性，并将邀请STEM学生帮助注释UAS图像。该项目将为机器人、计算机视觉/机器学习(CV/ML)、人机交互和地理空间土地使用社区管理佛罗里达州立大学和德克萨斯A&A&M大学在飓风伊恩期间部署无人驾驶航空系统(UAS)的车辆和飞行员数据。该项目有以下三个目标：1)整理数据(图像、日志文件、航班时刻表等)(2)与无人机飞行员单独和集体面谈，以捕捉人-机器人的性能、最佳做法、行为偏差和错误来源；以及3)分析任务日志和数据产品的性能(质量或完整性)，并记录飞行员随时间的质量、先前的培训以及在规范条件、操作节奏和疲劳条件下飞行任务的频率。从机器人学的角度来看，它将有助于新兴的模型，即在灾难期间如何使用多个代理，以及对设计、性能规范、人工智能的角色和无线通信的影响。这样的模式可以极大地提高国内无人机行业的竞争力，同时也可以激励蜂群研究的新方向。该项目的研究将为未来灾害的数据收集制定指导方针，为灾害工程和计算的进步奠定基础。它将增加CV/ML的训练数据的可用性，并作为从其他灾难中转移学习特征的试验台；这两者都可能导致机器学习的根本进步。从人的因素的角度来看，它将产生一种新的方法来创建人类-机器人数据集，将现场直接数据(没有现场实验者)与事件后数据相结合。这一方法将克服目前在极端工作环境中对科学问题进行实证调查方面的障碍，因为禁止嵌入实验者。这种方法预计将转移到其他极端工作环境，如核、太空、石油和天然气工业以及军事。人类-机器人数据本身可能导致在人为错误和劳动力培训方面的重大发现。从地理空间的角度来看，这些数据可以帮助确定海平面上升、建成环境以及之前风暴潮和洪水缓解的影响。总体而言，该项目将通过提高UAS在灾难后拯救生命和加快经济恢复的能力而造福社会，并有望创造出适用于极端环境的新技术的发现和方法。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This Grants for Rapid Response Research (RAPID) project will curate, supplement, and analyze data collected over a period of intensive uncrewed aerial system (UAS) operations, carried out as part of the State of Florida’s response to Hurricane Ian. From September 27, 2022, just before Hurricane Ian made landfall, and continually for the next nine days, teams from Florida State University and Texas A&M University helped coordinate 24 UAS pilots flying 16 different models of fixed-wing and rotorcraft UAS over Charlotte, Lee, and Hardee counties. These missions obtained aerial imagery to survey wind and flood damage, direct ground response, support strategic planning and resource allocation, monitor threats to public safety, and provide documentation for subsequent emergency relief funding. Under this award, the research team will curate 55,000 images and videos collected during the disaster, comprising over 750 gigabytes of data, and supporting material such as flight schedules and log files. The curated data, and derived products such as aerial maps and edited video, will be made publicly available for open-source use. The project will analyze the mission logs and data products to assess pilot performance over time, and will document variables potentially influencing pilot performance, including pilot skill, prior training and experience, operations tempo, and fatiguing conditions, supplemented by individual and collective interviews with the UAS pilots. The image dataset and derived products will help the computer vision/machine learning (CV/ML) community design better algorithms for identifying threats to public safety, damaged structures, and people in distress. The pilot performance dataset will be made available to the research community, to characterize human-robot performance, formulate best practices, and to understand deviation in behaviors and sources of mission error. The resulting insights into proper matching of vehicles, pilots, missions, and operational parameters will increase the ability of UAS platforms and pilots to save lives and accelerate economic recovery after a disaster. The datasets can help the domestic UAS industry improve products for response to a broad class of natural disasters, including wildfires and flooding, and for use in extreme environments, such as in oil and gas exploration and extraction and for in nuclear reactors and nuclear waste sites. The project will support the creation of better workflow procedures to reduce human error, increasing trust in the technology by UAS operators and other first responders, and facilitating adoption of UAS for emergency response. The project will broaden participation in science, with three women out of the four co-PIs, and will engage STEM students to help annotate the UAS imagery. This project will curate vehicle and pilot data from the deployment of uncrewed aerial systems (UAS) during Hurricane Ian by Florida State University and Texas A&M University for the robotics, computer vision/machine learning (CV/ML), human-robot interaction, and geospatial land-use communities. The project has the following three objectives: 1) Curate the data (imagery, log files, flight schedules, etc.) and data products (images, video, orthomosaic maps, digital surface maps) collected during the disaster and make available for open-source use; 2) Interview the UAS pilots individually and collectively in order to capture human-robot performance, best practices, deviation in behaviors, and sources of error; and 3) Analyze the mission logs and data products for performance (quality or completeness) and document the quality over time by pilots, prior training, and frequency of flying the missions in normative conditions, the operations tempo, and fatiguing conditions. From a robotics perspective, it will contribute to the emerging model of how multiple agents may be used during disasters, and the consequences for design, performance specifications, the role of artificial intelligence, and wireless communications. Such a model can greatly increase the competitiveness of the domestic drone industry, as well as motivate novel directions in swarm research. Research stemming from this project will generate guidelines for data collection in future disasters, setting the stage for advances in engineering and computing for disasters. It will increase the availability of training data for CV/ML and serve as a testbed for transfer of learned features from other disasters; both of which could lead to fundamental advances in machine learning. From a human-factors perspective, it will generate a new methodology for creating human-robot datasets that combine on-site direct data (with no experimenters in the field) with post-event data. This methodology would overcome current barriers in conducting empirical investigations into scientific questions on extreme work environments because of the prohibition on embedded experimenters. This methodology is expected to transfer to other extreme work environments, such as nuclear, space, oil and gas industry, and the military. The human-robot data itself could lead to major findings in human error and workforce training. From a geospatial perspective, the data can help establish the impact of rising sea levels, the built environment, and prior storm surge and flooding mitigations. Overall, the project will benefit society by increasing the ability to save lives and accelerate economic recovery after a disaster with UAS and is expected to create findings and methods that will generalize to new technologies for extreme environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.