FW-HTF: First Person View and Augmented Reality for Airborne Embodied Intelligent Cognitive Assistants

FW-HTF：机载智能认知助理的第一人称视角和增强现实

• 批准号: 1840044
• 负责人: Craig Woolsey
• 金额: $ 150万
• 依托单位: Virginia Polytechnic Institute and State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1840044&HistoricalAwards=false
• 关键词: FW; HTF; First; Person; View

The Future of Work at the Human-Technology Frontier (FW-HTF) is one of 10 new Big Ideas for Future Investment announced by NSF. The FW-HTF cross-directorate program aims to respond to the challenges and opportunities of the changing landscape of jobs and work by supporting convergent research. This award fulfills part of that aim. The future of work will involve human operators and semi-autonomous robotic systems. Human workers control the robots, extending their sensing and physical capabilities. This technology-empowered workforce will do remote, dangerous, and increasingly specialized work. An example use of this work might be the inspection of hard-to-access bridge supports. This project will explore the use of intelligent airborne drones to help ground-based operators in the inspection of highway bridges. Through the careful integration of augmented reality (AR) with first-person-view (FPV) operator interfaces this research will enhance worker performance across a wide range of otherwise very difficult tasks. The research program will address key challenges in the use of embodied intelligent cognitive assistants (e-ICAs) for infrastructure inspection. New principles of shared situation awareness will be developed for human/robot collaboration through AR/FPV user interfaces and these principles will inform interface design guidelines and evaluation measures. The research program will also emphasize collaborative perception and planning, such as peripheral/central computer vision to enhance shared situation awareness, gaze-informed adaptive viewpoint planning for improved image quality, and a global planning method for partially known and uncertain maps that adapts the plan in real-time as the worker discovers new information. Tunable control system performance will allow the worker and the e-ICA to collaborate in managing disturbance energy to optimize mission data quality and flight endurance while ensuring safety of flight. The parallel development of a public repository containing annotated deterioration imagery will support artificial intelligence based defect analytics to provide the human worker with real-time inspection cues. A parallel and coordinated economic and workforce analysis will assess the impact of airborne e-ICAs, controlled using FPV with augmented reality, on the future of work in infrastructure inspection and beyond.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类技术前沿的未来工作（FW-HTF）是NSF宣布的10个未来投资新想法之一。FW-HTF跨董事会计划旨在通过支持融合研究来应对不断变化的就业和工作环境的挑战和机遇。这个奖项实现了这个目标的一部分。未来的工作将涉及人类操作员和半自主机器人系统。人类工人控制机器人，扩展他们的感知和物理能力。这些技术赋能的劳动力将从事偏远、危险和日益专业化的工作。这项工作的一个例子可能是检查难以进入的桥梁支撑。该项目将探索使用智能空中无人机来帮助地面操作员检查公路桥梁。通过将增强现实（AR）与第一人称视角（FPV）操作界面仔细整合，这项研究将提高工人在各种其他非常困难的任务中的表现。该研究计划将解决使用嵌入式智能认知助理（e-ICA）进行基础设施检查的关键挑战。共享态势感知的新原则将通过AR/FPV用户界面为人类/机器人协作开发，这些原则将为界面设计指南和评估措施提供信息。该研究计划还将强调协作感知和规划，例如外围/中央计算机视觉，以增强共享的态势感知，基于凝视的自适应视点规划，以提高图像质量，以及用于部分已知和不确定地图的全局规划方法，该方法可以在工作人员发现新信息时实时调整计划。可调控制系统性能将允许工作人员和e-ICA在管理干扰能量方面进行协作，以优化使命数据质量和飞行耐力，同时确保飞行安全。包含注释退化图像的公共存储库的并行开发将支持基于人工智能的缺陷分析，为人类工作人员提供实时检查提示。一个并行和协调的经济和劳动力分析将评估机载e-ICA的影响，使用FPV与增强现实控制，对未来的工作在基础设施检查和超越。这个奖项反映了NSF的法定使命，并已被认为是值得支持的评估使用基金会的智力价值和更广泛的影响审查标准。

项目成果

A heavy sphere in a vertical jet: A simple pedagogical demonstration of aerodynamics and stability

垂直喷射中的重球：空气动力学和稳定性的简单教学演示

• DOI: 10.2514/6.2021-2802
• 发表时间: 2021
• 期刊: AIAA AVIATION 2021 Forum
• 作者: Swartzwelder, Zachary;Woolsey, Samuel;Woolsey, Craig A.
• 通讯作者: Woolsey, Craig A.

Cross-Track Control of Rotorcraft Using Passivity Based Techniques

使用基于无源技术的旋翼机交叉航迹控制

• DOI: 10.2514/6.2021-1991
• 发表时间: 2021
• 期刊: AIAA Scitech 2021 Forum
• 作者: Fahmi, Jean-Michel W.;Woolsey, Craig A.
• 通讯作者: Woolsey, Craig A.

Augmented Reality for Infrastructure Inspection with Semi-autonomous Aerial Systems: An Examination of User Performance, Workload, and System Trust

使用半自主航空系统进行基础设施检查的增强现实：用户性能、工作负载和系统信任的检查

• DOI: 10.1109/vrw50115.2020.00222
• 发表时间: 2020
• 期刊: IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW
• 作者: Dam, Jared Van;Krasner, Alexander;Gabbard, Joseph L.
• 通讯作者: Gabbard, Joseph L.

COCO-Bridge: Structural Detail Data Set for Bridge Inspections

• DOI: 10.1061/(asce)cp.1943-5487.0000949
• 发表时间: 2021-05-01
• 期刊: JOURNAL OF COMPUTING IN CIVIL ENGINEERING
• 影响因子: 6.9
• 作者: Bianchi, Eric;Abbott, Amos Lynn;Hebdon, Matthew
• 通讯作者: Hebdon, Matthew

View Planning and Navigation Algorithms for Autonomous Bridge Inspection with UAVs

查看无人机自主桥梁检查的规划和导航算法

• DOI: 10.1007/978-3-030-33950-0_18
• 发表时间: 2020
• 期刊: International Symposium on Experimental Robotics
• 作者: P. Shanthakumar1, K. Yu