FW-HTF-RM: Collaborative Research: Assistive Intelligence for Cooperative Robot and Inspector Survey of Infrastructure Systems (AI-CRISIS)

美国民用基础设施面临着未来迅速恶化的前景。例如，美国60多万座公路桥中有39%已经超过了50年的设计寿命。为了确保市民的安全和福利，州交通部门被要求每两年检查一次桥梁。为了使检查和报告的客观性和一致性，最近引入了无人机等机器人来进行自动桥梁调查。这些配备了摄像头的机器人将迅速提供大量的调查数据，这些数据可用于帮助检查员评估桥梁的状况。为了将这些多学科课题转化为新的综合桥梁检测能力，这个由多所大学组成的团队将开发和实施一个具有辅助智能（AI）的协作机器人检查员系统，以使未来的桥梁检测更快、更便宜、更安全、更一致。机器人平台配备了红外摄像机和具有智能算法的中央处理单元，将以飞行和爬行两种模式运行，接近桥梁的各个部分/元素，并收集整个桥梁的高保真图像。检查员将使用一套可穿戴传感器进行仪器和监控，以在每次桥梁检查期间实现最佳的机器人检查员合作。人工智能算法将在模块中开发，以分析来自人体传感器和摄像头的大数据，以支持检查前的劳动力培训，检查中元素缺陷检测和桥梁检查后的状态评估。机器人-检查员-人工智能系统将与利益相关者合作，在六个桥址进行整合和验证。这个多学科项目旨在探索和发展人工智能的科学知识和底层方法，以实现检查员-机器人对桥梁的合作调查。工作范围包括：(1)利用主动热成像和可见光成像快速准确地检测钢筋混凝土板（代表性桥梁构件）的表面和内部缺陷，开发一种最佳的从筛选到探测的检测策略；(2)建立检查员任务绩效的因果模型，以支持优化工作设计、有效培训和现场操作；(3)开发有人工检查员参与的自适应人工智能算法，以快速适应未来的工作环境，并进行高效可靠的数据分析；(4)最大限度地利用领域专业知识，实现整座桥梁要素之间的空间相关性，从而提高自适应人工智能的有效性和适用性。本研究中开发的方法和技术将通过出版物、演讲、现场演示和培训，转移到利益相关者、研究人员、政策制定者和最终用户（桥梁检查员）手中。特别是，世界上第一个桥梁基准数据集，包括视频、标记物体和从真实桥梁建立的元数据，经利益相关者批准后，将通过人工智能桥梁检查比赛与国际社会共享。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The U.S. civil infrastructure faces the prospect of rapid future deterioration. For example, 39% of the over 600,000 highway bridges in the U.S. have exceeded their design life of 50 years. To ensure the safety and welfare of citizens, state Departments of Transportation are required to inspect bridges every two years. To make inspection and reporting objective and consistent, robots such as unmanned aerial vehicles have recently been introduced to perform autonomous surveys of bridges. These robots, equipped with cameras, will rapidly provide a large set of survey data that can be used to aid inspectors in assessing the condition of bridges. To transform these multidisciplinary topics into a new integrated bridge inspection capability, this multi-university team will develop and implement a cooperative robot-inspector system with assistive intelligence (AI) in order to make the future bridge inspection significantly faster, cheaper, safer, and more consistent. A robotic platform equipped with infrared cameras and a central processing unit with intelligent algorithms will operate in both flying and crawling modes, travel in proximity to various parts/elements of a bridge, and collect high-fidelity images of the entire bridge. Inspectors will be instrumented and monitored with a suite of wearable sensors to enable optimal robot-inspector cooperation during each bridge inspection. AI algorithms will be developed in modules to analyze the big data from human sensors and cameras in support of the pre-inspection workforce training, during-inspection element defect detection, and post-inspection condition evaluation of bridges. The robot-inspector-AI system will be integrated and validated at six bridge sites in collaboration with stakeholders.This multidisciplinary project aims to explore and develop the scientific knowledge and underling methods of AI for a cooperative inspector-robot survey of bridges. The scope of work includes: (1) to develop an optimal screening-to-probing inspection strategy using fast and accurate detection of both surface and internal defects in reinforced concrete slabs (representative bridge elements) from active thermal and visible light imaging; (2) to create a causal model of inspectors’ task performance to support an optimal job design, effective training, and onsite operation; (3) to develop adaptive AI algorithms with human inspectors in the loop for a rapid adaptation to future work contexts, and efficient and reliable data analyses; and (4) to enable a spatial correlation among the elements of an entire bridge with a maximum use of domain expertise, and thus improve the effectiveness and applicability of adaptive AI. The methods and technologies developed in this study will be transferred into the hands of stakeholders, researchers, policy makers, and end users (bridge inspectors) through publications, presentations, field demonstrations, and training. In particular, the world’s first bridge benchmark dataset of videos, labeled objects, and metadata established from real-world bridges, upon approval by stakeholders, will be shared with the international community via an AI-enabled bridge inspection competition.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.