Automated content-based filtering for enhanced vision-based documentation in construction toward exploiting big visual data from drones
Automated content-based filtering for enhanced vision-based documentation in construction toward exploiting big visual data from drones
复制标题
自动化基于内容的过滤，用于增强建筑中基于视觉的文档，以利用无人机的大视觉数据
DOI：
10.1016/j.autcon.2019.102831
复制
发表时间：
2019-09
影响因子：
10.3
通讯作者：
Youngjib Ham;M. Kamari
中科院分区：
文献类型：
--
作者：
Youngjib Ham;M. Kamari
近年来，新兴的移动的设备和配备摄像头的平台为视觉捕捉和不断记录建筑工地的现状提供了极大的便利。在这方面，经常以大量照片或长视频的形式收集视觉数据。然而，从工地收集的大量视觉数据（例如，通过无人驾驶飞行器（UAV）每天或每周进行数据收集）已经引起视觉数据过载，这是一个不可避免的问题。针对建筑领域的数据过载问题，提出了一种新的方法来自动检索分散在采集的视频片段或连续图像中的包含建筑相关内容的具有照片价值的帧。在所提出的方法中，感兴趣对象的存在（即，通过语义分割来识别给定图像帧中的对象（例如，与构造相关的内容），然后基于所识别的对象的空间组成来计算图像帧的分数。为了提高过滤性能，高分图像帧被进一步分析以估计它们被有意拍摄的可能性。在两个建筑工地的案例研究表明，所提出的方法的准确性是接近人类的判断过滤视觉数据检索照片值得包含施工相关内容的图像帧。性能指标表明，语义分割的准确率约为91%，与以前的作品相比，我们观察到在过滤建筑视觉数据时增强了类似人类的判断。预计拟议的自动化方法使从业者能够通过选择性的视觉数据有效地评估建筑工地的现状，从而促进在正确的时间做出数据驱动的决策。
In recent years, emerging mobile devices and camera-equipped platforms have offered a great convenience to visually capture and constantly document the as-is status of construction sites. In this regard, visual data are regularly collected in the form of numerous photos or lengthy videos. However, massive amounts of visual data that are being collected from jobsites (e.g., data collection on daily or weekly bases by Unmanned Aerial Vehicles, UAVs) has provoked visual data overload as an inevitable problem to face. To address such data overload issue in the construction domain, this paper aims at proposing a new method to automatically retrieve photo-worthy frames containing construction-related contents that are scattered in collected video footages or consecutive images. In the proposed method, the presence of objects of interest (i.e., construction-related contents) in given image frames are recognized by the semantic segmentation, and then scores of the image frames are computed based on the spatial composition of the identified objects. To improve the filtering performance, high-score image frames are further analyzed to estimate their likelihood to be intentionally taken. Case studies in two construction sites have revealed that the accuracy of the proposed method is close-to-human judgment in filtering visual data to retrieve photo-worthy image frames containing construction-related contents. The performance metrics demonstrate around 91% of accuracy in the semantic segmentation, and we observed enhanced human-like judgment in filtering construction visual data comparing to prior works. It is expected that the proposed automated method enables practitioners to assess the as-is status of construction sites efficiently through selective visual data, thereby facilitating data-driven decision making at the right time.