AR-integrated intelligent visual inspection system for health monitoring of constructed facilities

AR集成智能视觉检测系统，用于建筑设施的健康监测

• 批准号: RGPIN-2022-05151
• 负责人: Chen, PoHan
• 金额: $ 2.26万
• 依托单位: Concordia University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=746377
• 关键词: AR; integrated; intelligent; visual; inspection

The aging of constructed facilities, such as buildings, bridges, pipelines, etc., is a critical issue worldwide, as it affects the safety of the facilities. In Canada, many constructed facilities, especially infrastructure facilities, are in sparsely populated or indigenous areas, and the development of an automated intelligent visual inspection system that can greatly reduce manpower requirement and enhance inspection efficiency and effectiveness is necessary. In this regard, the goal of this research program is to develop near-real-time (NRT) fast defect recognition algorithms and utilize augmented reality (AR) to develop an AR-integrated intelligent visual inspection system for near-real-time health monitoring of constructed facilities. This intelligent system will comprise two modes: (1) automated drone inspection mode (M1); and (2) in-person inspection mode (M2). For M1, a GPS-guided drone will fly along a pre-planned route to automatically capture facility surface inspection data (e.g., videos or photos), and display them on a tablet or smartphone in real time, followed by immediate superimposing of recognized defects on the displayed view (i.e., AR effects). Whenever a defect is identified, the corresponding GPS position will be recorded. After the drone finishes the entire planned route, it will automatically fly back to each of the recorded positions to capture more detailed data around the spotted defect. For M2, an AR-HMD (head-mounted display) will capture inspection data and transmit them to a laptop for fast defect recognition, followed by superimposing recognized defects on AR-HMD. Interactive gesture control of AR-HMD will be incorporated in the development. To achieve the goal of this program, four research objectives are set: (1) to develop near-real-time fast defect recognition algorithms for steel and reinforced concrete (RC) constructed facilities; (2) to integrate a drone with tablets/smartphones of both iOS/iPadOS and Android systems for the M1 mode; (3) to integrate an AR-HMD and its gesture control with a laptop for the M2 mode; and (4) to design and develop user-friendly AR user interfaces for both modes. Nine Highly Qualified Personnel (HQP) will be trained in this program, including 3 PhD students, 4 MASc students, and 2 undergraduate research assistants (URAs). Nowadays, the AR applications in the Architecture, Engineering and Construction (AEC) industry mostly rely on pre-loaded data on the AR equipment for object or pattern recognition. This program will bring AR applications to the next level by processing newly captured data in a near-real-time manner, which leads to the importance of the fast defect recognition algorithms to be developed. At the same time, the proposed automated drone inspection mode and the in-person inspection mode with interactive AR-HMD may both revolutionize the current inspection practices and significantly benefit the public and private engineering sectors in Canada and beyond.

建筑物、桥梁、管道等已建设施的老化，是一个全球性的关键问题，因为它影响到设施的安全。在加拿大，许多已建成的设施，特别是基础设施，都位于人口稀少或土著地区，因此有必要开发一种自动化智能视觉检测系统，可以大大减少人力需求，提高检测效率和有效性。在这方面，该研究计划的目标是开发近实时（NRT）快速缺陷识别算法，并利用增强现实（AR）开发一个AR集成的智能视觉检测系统，用于近实时的健康监测。该智能系统将包括两种模式：（1）自动无人机检查模式（M1）;以及（2）亲自检查模式（M2）。 对于M1，GPS引导的无人机将沿着预先规划的路线飞行，以自动捕获设施表面检查数据（例如，视频或照片），并将它们真实的实时显示在平板电脑或智能手机上，随后将识别的缺陷立即叠加在所显示的视图上（即，AR效应）。一旦发现缺陷，将记录相应的GPS位置。无人机完成整个计划路线后，它将自动飞回每个记录的位置，以捕获发现缺陷周围的更详细数据。对于M2，AR-HMD（头戴式显示器）将捕获检测数据并将其传输到笔记本电脑进行快速缺陷识别，然后将识别的缺陷叠加在AR-HMD上。AR-HMD的交互式手势控制将被纳入开发中。为了实现该计划的目标，设定了四个研究目标：（1）开发用于钢和钢筋混凝土（RC）结构设施的近实时快速缺陷识别算法;（2）将无人机与iOS/iPadOS和Android系统的平板电脑/智能手机集成为M1模式;（3）将AR-HMD及其手势控制与笔记本电脑集成为M2模式;（4）将AR-HMD及其手势控制与笔记本电脑集成为M1模式。以及（4）为两种模式设计和开发用户友好的AR用户界面。9名高素质人员（HQP）将在这个计划中培养，包括3名博士生，4名MASc学生和2名本科生研究助理（URA）。如今，建筑、工程和建筑（AEC）行业中的AR应用主要依赖于AR设备上的预加载数据来进行对象或模式识别。该计划将通过以近乎实时的方式处理新捕获的数据，将AR应用提升到一个新的水平，这导致了快速缺陷识别算法的重要性。与此同时，拟议的自动无人机检查模式和带有交互式AR-HMD的亲自检查模式可能会彻底改变目前的检查实践，并使加拿大及其他地区的公共和私营工程部门受益匪浅。