Semantic Image Retrieval and Clustering for Supporting Domain-Specific Bridge Component and Defect Classification
Semantic Image Retrieval and Clustering for Supporting Domain-Specific Bridge Component and Defect Classification
复制标题
用于支持特定领域桥梁构件和缺陷分类的语义图像检索和聚类
DOI：
10.1061/9780784482858.087
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
El-Gohary, Nora
中科院分区：
文献类型：
--
作者：
Liu, Peter Cheng-Yang;El-Gohary, Nora
从图像中自动检测和分类缺陷对于桥梁劣化预测和维护决策变得越来越重要。大多数现有的缺陷检测工作已经开发了用于训练机器学习算法以进行检测/分类的数据集。然而，这些数据集的大多数都受到两个主要限制。首先，大多数数据集的大小都相对较小，这不足以构建一个经过良好训练的准确图像分类器。其次，大多数数据集缺乏场景、角度和背景的多样性，无法适应不同的应用场景和环境。为了解决这些局限性，本文提出了一种语义图像检索和聚类方法，收集了大量的相关图像与各种场景，角度，和背景从Web和聚类这些图像支持特定领域的桥梁组件和缺陷检测。该方法包括三个主要步骤：查询形成和图像搜索和检索，图像表示，和图像聚类。首先，从桥梁检测文档中提取一组特定领域的单词，并将其用作从Web中检索大量图像的查询。其次，使用迁移学习技术将用于一般图像分类的预训练模型中的知识迁移到桥组件和缺陷相关图像聚类任务。使用具有预训练权重的深度卷积神经网络（CNN）来提取图像的视觉特征以用于图像表示。第三，聚类技术被用来聚类图像的基础上提取的特征。该方法的性能进行了评估，使用轮廓系数。评估结果表明，该方法是有前途的。
Automatic defect detection and classification from images is becoming increasingly important for bridge deterioration prediction and maintenance decision making. The majority of existing defect detection efforts have developed their datasets for training a machine-learning algorithm for detection/classification. However, the majority of these datasets suffer from two main limitations. First, most of the datasets are relatively small in size, which is not sufficient to build a well-trained, accurate image classifier. Second, most of the datasets lack the needed variety in scenes, angles, and backgrounds, which is not adaptable to different application contexts and environments. To address these limitations, this paper proposes a semantic image retrieval and clustering method to collect a large size of relevant images with various scenes, angles, and backgrounds from the Web and cluster these images for supporting domain-specific bridge component and defect detection. The proposed method includes three primary steps: query formation and image search and retrieval, image representation, and image clustering. First, a set of domain-specific words were extracted from bridge inspection documents and used as queries for retrieving a large number of images from the Web. Second, a transfer learning technique was used to transfer knowledge in a pre-trained model for general image classification to the bridge component and defect-related image clustering task. A deep convolutional neural network (CNN) with pre-trained weights was used to extract the visual features of the images for image representation. Third, a clustering technique was used to cluster the images based on the extracted features. The performance of the proposed method was evaluated using the silhouette coefficient. The evaluation results show that the proposed method is promising.