Deep Neural Network based Visual Inspection with 3D Metric Measurement of Concrete Defects using Wall-climbing Robot
Deep Neural Network based Visual Inspection with 3D Metric Measurement of Concrete Defects using Wall-climbing Robot
复制标题
DOI：
10.1109/iros40897.2019.8968195
复制
发表时间：
2019-11
期刊：
影响因子：
--
通讯作者：
Liang Yang;Bing Li;Guoyong Yang;Yong Chang;Zhaoming Liu;Biao Jiang;Jizhong Xiao
中科院分区：
文献类型：
--
作者：
Liang Yang;Bing Li;Guoyong Yang;Yong Chang;Zhaoming Liu;Biao Jiang;Jizhong Xiao
本文提出了一种新型的度量检测机器人系统，该系统使用深度神经网络来检测和测量表面缺陷（即，裂缝和剥落）的混凝土结构上执行的爬壁机器人。该系统由四个模块组成：机器人数据采集模块用于获取RGB-D图像和IMU测量结果，视觉-惯性SLAM模块用于生成具有深度信息的位姿耦合关键帧，InspectionNet模块用于将每个像素分为三类（背景、裂纹和剥落），以及3D配准和地图融合模块，用于将缺陷片配准到与检测到的缺陷重叠并突出显示的配准的3D模型中，上下文可视化。该系统使每个表面缺陷补丁的度量模型具有像素级精度，并确定其在3D空间中的位置，这对于结构健康评估和监测是重要的。InspectionNet在裂纹和剥落检测方面的平均准确率达到87.64%。我们还证明了我们的InspectionNet对视角，尺度和照明变化具有鲁棒性。最后，我们设计了一个度量体素体地图，突出的三维模型中的缺陷，并提供位置和度量信息。
This paper presents a novel metric inspection robot system using a deep neural network to detect and measure surface flaws (i.e., crack and spalling) on concrete structures performed by a wall-climbing robot. The system consists of four modules: robotics data collection module to obtain RGB-D images and IMU measurement, visual-inertial SLAM module to generate pose coupled key-frames with depth information, InspectionNet module to classify each pixel into three classes (back-ground, crack and spalling), and 3D registration and map fusion module to register the flaw patch into registered 3D model overlaid and highlighted with detected flaws for spatial-contextual visualization. The system enables the metric model of each surface flaw patch with pixel-level accuracy and determines its location in 3D space that is significant for structural health assessment and monitoring. The InspectionNet achieves an average accuracy of 87.64% for crack and spalling inspection. We also demonstrate our InspectionNet is robust to view angle, scale and illumination variation. Finally, we design a metric voxel volume map to highlight the flaw in 3D model and provide location and metric information.