Deep learning based structural health monitoring with autonomous UAVs

加拿大迫切需要一个自动化、可靠、高效、稳健的健康监测系统来检测老化桥梁系统中受损的基础设施。许多基于振动的方法都是利用接触式传感器来检测桥梁损伤的。然而，这些方法不可靠且昂贵，因为它们容易受到不确定性、噪声和环境变化的影响，并且需要安装许多传感器。由训练有素的工程师进行目视检查是目前的主要方法，但也很昂贵，一年两次的检查不能充分防止突然坍塌。近年来，人们提出了基于计算机视觉（CV）的损伤检测方法来支持视觉检测。目前，该方法仅限于检测一种损伤类型。此外，利用传统图像处理和传统机器学习提取的弱损伤敏感特征对损伤特征进行分类。因此，它对输入图像的光照条件的变化不具有鲁棒性。自2017年以来，我开创了基于深度学习（DL）的损伤检测，以克服传统基于简历的方法的局限性，通过我以前的NSERC发现资助。这项研究引起了全世界的兴趣，并进行了许多后续研究。我还发起了一种自主无人机飞行方法的开发，使用超声波信标系统（UBS）在全球定位系统（GPS）拒绝的区域进行飞行。这些区域包括无法使用GPS的桥面下方空间，以及重要结构构件及其连接的安全关键区域。然而，开发一个完全自动化的桥梁检查和管理系统需要广泛的调查。这样一个系统是拟议研究计划的长期目标。本提案的短期目标是：(1)开发基于先进DL方法的像素级多重损伤识别，使用常规和热像仪对外部和内部损伤进行识别；(2)开发一种不使用GPS或UBS的整个桥梁系统的自主飞行方法；(3)开发一种整体三维（3D）损伤映射方法，用于有效的桥梁管理；(4)整合所有已开发的方法，用于全自动桥梁检测系统。这种自动检测系统将是可靠的，准确的，并通过提供一个明确的，可视化的，整体的三维损伤图，有效的桥梁管理成本效益。自动化可以频繁检查，并可以通过检测早期的内部和外部损坏来防止突然崩溃。这项研究将产生一个自动桥梁检测系统的原型，这将与人工智能和全球机电行业相关。原型机将使用DL方法和具有无线自动充电功能的自主飞行无人机系统进行开发。因此，马尼托巴大学在这一研究课题上继续处于世界领先地位。

An automated reliable, efficient, and robust health monitoring system is urgently needed to detect damaged infrastructure in Canada's aging bridge system. Many vibration-based methods have been developed to detect bridge damage using contact sensors. However, these methods are unreliable and expensive, because they are vulnerable to uncertainties, noise, and environmental changes and require many installed sensors. Visual inspection by trained engineers is the current main approach, but is also costly, and biannual inspection cannot adequately prevent sudden collapse. Recently, computer vision (CV)-based damage detection methods have been proposed to support visual inspection. At present, this method is limited to detecting only one damage type. Additionally, it uses weak damage sensitive features extracted from traditional image processing and traditional machine learning to classify damage features. Thus, it is not robust to changes in the lighting conditions of input images. Since 2017, I have pioneered deep learning (DL)-based damage detection to overcome the limitations of traditional CV-based approaches through my previous NSERC Discovery Grant. This research has received worldwide interest, and numerous follow-up studies have been conducted. I also initiated the development of an autonomous drone flight method to enable flight in global positioning system (GPS) denied areas using an ultrasonic beacon system (UBS). These areas include the space beneath a bridge deck, where GPS is unavailable, and safety-critical areas, where important structural members and their connections are found. However, extensive investigation is required to develop a fully automated bridge inspection and management system. Such a system is the long-term goal of the proposed research program. The short-term objectives of this proposal are to (1) develop pixel-level multiple-damage identification based on advanced DL methods for external and internal damage using regular and thermal cameras, (2) develop an autonomous flight method for an entire bridge system without using GPS or UBS, (3) develop a holistic three-dimensional (3D) damage mapping method for efficient bridge management, and (4) integrate of all the developed methods for the fully automated bridge inspection system. This automated inspection system will be reliable, accurate, and cost-efficient by providing an explicit, visual, and holistic 3D damage map for efficient bridge management. Automation enables frequent inspection and can prevent sudden collapse by detecting early-stage internal and external damage. This research will produce a prototype of the automated bridge inspection system, which will be relevant to artificial intelligence and mechanical and electrical industries worldwide. The prototype will be developed using a DL method and the autonomous flight drone system with wireless auto-charging. Accordingly, University of Manitoba continues to be a world leader in this research topic.