Deep learning based structural health monitoring with autonomous UAVs

迫切需要一个自动化的可靠，高效和强大的健康监测系统来检测加拿大老化桥梁系统中受损的基础设施。许多基于振动的方法已经被开发来使用接触传感器检测桥梁损伤。然而，这些方法不可靠且昂贵，因为它们容易受到不确定性、噪声和环境变化的影响，并且需要安装许多传感器。由受过训练的工程师进行目视检查是目前的主要方法，但费用也很高，而且一年两次的检查不足以防止突然倒塌。最近，已经提出了基于计算机视觉（CV）的损伤检测方法来支持视觉检测。目前，该方法仅限于检测一种损伤类型。此外，它使用传统的图像处理和传统的机器学习提取的弱损伤敏感特征进行分类损伤特征。因此，对于输入图像的照明条件的变化不鲁棒。自2017年以来，我通过之前的NSERC Discovery Grant开创了基于深度学习（DL）的损伤检测，以克服传统基于CV的方法的局限性。这项研究受到了全世界的关注，并进行了许多后续研究。我还发起了一种自主无人机飞行方法的开发，以便使用超声波信标系统（UBS）在全球定位系统（GPS）拒绝的区域飞行。这些区域包括无法使用GPS的桥面下方空间，以及发现重要结构构件及其连接的安全关键区域。然而，需要进行广泛的调查，以开发全自动桥梁检查和管理系统。这样一个系统是拟议研究计划的长期目标。本提案的短期目标是：（1）使用常规和热成像摄像机，基于外部和内部损伤的先进DL方法，开发像素级多损伤识别;（2）开发一种不使用GPS或UBS的整个桥梁系统的自主飞行方法;（3）开发一种整体三维（3D）损伤映射方法，用于有效的桥梁管理，（4）将已开发的各种方法集成到全自动桥梁检测系统中。 该自动化检测系统将通过提供明确、可视化和整体的3D损伤图来实现可靠、准确和经济高效的桥梁管理。自动化可实现频繁检查，并可通过检测早期内部和外部损坏来防止突然倒塌。这项研究将产生一个自动化桥梁检测系统的原型，这将与人工智能和机械和电气行业有关。该原型将使用DL方法和具有无线自动充电的自主飞行无人机系统进行开发。因此，马尼托巴大学继续在这一研究课题的世界领先地位。

An automated reliable, efficient, and robust health monitoring system is urgently needed to detect damaged infrastructure in Canada's aging bridge system. Many vibration-based methods have been developed to detect bridge damage using contact sensors. However, these methods are unreliable and expensive, because they are vulnerable to uncertainties, noise, and environmental changes and require many installed sensors. Visual inspection by trained engineers is the current main approach, but is also costly, and biannual inspection cannot adequately prevent sudden collapse. Recently, computer vision (CV)-based damage detection methods have been proposed to support visual inspection. At present, this method is limited to detecting only one damage type. Additionally, it uses weak damage sensitive features extracted from traditional image processing and traditional machine learning to classify damage features. Thus, it is not robust to changes in the lighting conditions of input images. Since 2017, I have pioneered deep learning (DL)-based damage detection to overcome the limitations of traditional CV-based approaches through my previous NSERC Discovery Grant. This research has received worldwide interest, and numerous follow-up studies have been conducted. I also initiated the development of an autonomous drone flight method to enable flight in global positioning system (GPS) denied areas using an ultrasonic beacon system (UBS). These areas include the space beneath a bridge deck, where GPS is unavailable, and safety-critical areas, where important structural members and their connections are found. However, extensive investigation is required to develop a fully automated bridge inspection and management system. Such a system is the long-term goal of the proposed research program. The short-term objectives of this proposal are to (1) develop pixel-level multiple-damage identification based on advanced DL methods for external and internal damage using regular and thermal cameras, (2) develop an autonomous flight method for an entire bridge system without using GPS or UBS, (3) develop a holistic three-dimensional (3D) damage mapping method for efficient bridge management, and (4) integrate of all the developed methods for the fully automated bridge inspection system. This automated inspection system will be reliable, accurate, and cost-efficient by providing an explicit, visual, and holistic 3D damage map for efficient bridge management. Automation enables frequent inspection and can prevent sudden collapse by detecting early-stage internal and external damage. This research will produce a prototype of the automated bridge inspection system, which will be relevant to artificial intelligence and mechanical and electrical industries worldwide. The prototype will be developed using a DL method and the autonomous flight drone system with wireless auto-charging. Accordingly, University of Manitoba continues to be a world leader in this research topic.