Development and Evaluation of Real-like Synthetic Construction Images to Enhance the Performances of Deep Neural Network for Construction Applications

深度神经网络(DNN)现在被视为建筑机器人人工智能未来的核心。然而，建筑业在机器人技术方面的进步远远落后于汽车和制造业等其他行业。这些问题-建筑工地极其动态、快速变化和拥挤-对建筑机器人的部署构成了极端的挑战。DNN支持的人工智能被视为开启人类对建筑工地理解的关键。但DNN模型需要大量的图像进行训练，才能获得最佳性能。而这些大的训练集根本不存在于建筑领域。如果没有这样的训练数据，结果是一个有偏见的过拟合模型，精度和可扩展性都很低。在此背景下，我的短期研究目标是通过采用一种利用非真实但看起来真实的虚拟建筑图像的新颖训练方法来提高基于DNN的视觉场景理解模型的准确性和可扩展性。这一短期目标是开发人工智能技术的关键一步，该技术实现了对建筑背景的人类水平的视觉场景理解。如果我们能够在不需要人工输入的情况下自动合成、生成和标记建筑工地图像，使DNN能够通过无限数量的训练数据达到更高的性能，并允许探索更深层次的DNN架构，这将是具有开创性的。共享非真实数据将不存在法律问题；允许此类数据被用作研究小组之间公平竞争和合作的基础。为此，我的短期研究目标是：1)开发一个计算框架，可以自动合成和标记非真实但看起来真实的建筑图像，这些图像捕捉虚拟建筑工人的化身。2)建立微型基础设施，可以记录和标记非真实但看起来真实的建筑图像，这些图像捕捉微型建筑设备。3)描述了非真实图像对DNN训练的影响，回答了以下问题：(I)非真实建筑图像对改进现有DNN模型是否有效？以及(Ii)在DNN训练中，非真实的建筑图像能产生与真实图像相同的效果吗？我的短期研究成果是将建筑图像的训练数据集快速有效地扩展了几个数量级，对建筑领域的DNN研究产生了多个重要影响，包括：(I)建筑场景理解的DNN模型的显著改进；(Ii)显著节省了时间和资源；以及(Iii)使跨建筑研究的集体知识发现成为可能。未来的建筑机器人将安全地与现场工人合作，显著提高生产率，并抵消日益严重的劳动力短缺。拟议的研究对实现这一愿景至关重要。

Deep neural networks (DNNs) are now seen as central to the future of artificial intelligence for construction robots. However, progress in robotics for the construction industry lags far behind that of other industries such as automobile and manufacturing. The problem-construction sites are extremely dynamic, rapidly changing, and crowded-poses an extreme challenge to the deployment of construction robots. DNN-powered AI is seen as the key to unlocking human-level understanding of construction sites. But DNN models need large sets of images to train on before they can obtain optimum performance. And these large training sets simply do not exist in the construction domain. Absent such training data, the outcome is a biased overfitted model with low accuracy and scalability. Against this backdrop, my short-term research goal is to improve the accuracy and scalability of DNN-based visual scene understanding models by deploying a novel training method that leverages non-real but real-looking virtual construction images. This short-term goal is an essential step in developing AI technology that achieves human-level visual scene understanding for construction contexts. It will be ground-breaking if we can automatically synthesize, generate, and label construction site images with no manual inputs, allowing DNNs to reach higher performance through a limitless quantity of training data, and allowing for the exploration of even deeper DNN architectures. There will be no legal issues in sharing non-real data; allowing such data to be used as the basis for fair competition and collaboration across research groups. To this end, my short-term research objectives are to: 1) Develop a computational framework that can automatically synthesize and label non-real but real-looking construction images that capture virtual construction worker avatars. 2) Build miniature-scale infrastructure that can record and label non-real but real-looking construction images that capture miniature-scale construction equipment. 3) Characterize the effect of the non-real images on a DNN's training, answering the following questions: (i) are the non-real construction images effective in improving the existing DNN models? and (ii) can a non-real construction image result in the same effect as a real image on DNN training? My short-term research outcome is to rapidly and efficiently expand training datasets of construction images by several orders of magnitude, with multiple important impacts on the study of DNNs in the construction domain, including: (i) dramatic improvements in DNN models for construction scene understanding; (ii) significant savings in time and resources; and (iii) enabling collective knowledge discovery across construction studies. Future construction robots will collaborate with field workers safely, improving productivity significantly and offsetting the growing labour shortage. The proposed research is essential to realizing this vision.