Development and Evaluation of Real-like Synthetic Construction Images to Enhance the Performances of Deep Neural Network for Construction Applications

深度神经网络（dnn）现在被视为建筑机器人人工智能未来的核心。然而，建筑行业的机器人技术的进步远远落后于汽车和制造业等其他行业。建筑工地是动态的、快速变化的、拥挤的，这对建筑机器人的部署提出了极大的挑战。深度神经网络驱动的人工智能被视为解锁人类对建筑工地理解的关键。但DNN模型需要大量的图像集来训练，才能获得最佳性能。而这些大型训练集在构造领域根本不存在。没有这样的训练数据，结果是一个有偏的过拟合模型，精度和可扩展性都很低。在此背景下，我的短期研究目标是通过部署一种利用非真实但看起来真实的虚拟建筑图像的新颖训练方法来提高基于dnn的视觉场景理解模型的准确性和可扩展性。这个短期目标是开发人工智能技术的重要一步，该技术可以实现对建筑环境的人类级别的视觉场景理解。如果我们能够在没有人工输入的情况下自动合成、生成和标记建筑工地图像，允许DNN通过无限大的训练数据达到更高的性能，并允许探索更深入的DNN架构，这将是突破性的。共享非真实数据不会产生法律问题；允许这些数据被用作跨研究小组公平竞争和合作的基础。为此，我的短期研究目标是：1)开发一个计算框架，可以自动合成和标记非真实但看起来真实的建筑图像，捕捉虚拟建筑工人的头像。2)建立微型规模的基础设施，可以记录和标记捕捉微型规模施工设备的非真实但看起来真实的施工图像。3)描述非真实图像对DNN训练的影响，回答以下问题：(i)非真实构造图像对改进现有DNN模型是否有效？（ii）在DNN训练中，非真实的构造图像能否产生与真实图像相同的效果？我的短期研究成果是快速有效地将建筑图像的训练数据集扩展几个数量级，这对建筑领域的DNN研究产生了多重重要影响，包括：(i)用于建筑场景理解的DNN模型的显着改进；（ii）显著节省时间和资源；（iii）在建筑研究中实现集体知识发现。未来的建筑机器人将与现场工人安全地合作，显著提高生产率，抵消日益严重的劳动力短缺。拟议的研究对实现这一愿景至关重要。

Deep neural networks (DNNs) are now seen as central to the future of artificial intelligence for construction robots. However, progress in robotics for the construction industry lags far behind that of other industries such as automobile and manufacturing. The problem-construction sites are extremely dynamic, rapidly changing, and crowded-poses an extreme challenge to the deployment of construction robots. DNN-powered AI is seen as the key to unlocking human-level understanding of construction sites. But DNN models need large sets of images to train on before they can obtain optimum performance. And these large training sets simply do not exist in the construction domain. Absent such training data, the outcome is a biased overfitted model with low accuracy and scalability. Against this backdrop, my short-term research goal is to improve the accuracy and scalability of DNN-based visual scene understanding models by deploying a novel training method that leverages non-real but real-looking virtual construction images. This short-term goal is an essential step in developing AI technology that achieves human-level visual scene understanding for construction contexts. It will be ground-breaking if we can automatically synthesize, generate, and label construction site images with no manual inputs, allowing DNNs to reach higher performance through a limitless quantity of training data, and allowing for the exploration of even deeper DNN architectures. There will be no legal issues in sharing non-real data; allowing such data to be used as the basis for fair competition and collaboration across research groups. To this end, my short-term research objectives are to: 1) Develop a computational framework that can automatically synthesize and label non-real but real-looking construction images that capture virtual construction worker avatars. 2) Build miniature-scale infrastructure that can record and label non-real but real-looking construction images that capture miniature-scale construction equipment. 3) Characterize the effect of the non-real images on a DNN's training, answering the following questions: (i) are the non-real construction images effective in improving the existing DNN models? and (ii) can a non-real construction image result in the same effect as a real image on DNN training? My short-term research outcome is to rapidly and efficiently expand training datasets of construction images by several orders of magnitude, with multiple important impacts on the study of DNNs in the construction domain, including: (i) dramatic improvements in DNN models for construction scene understanding; (ii) significant savings in time and resources; and (iii) enabling collective knowledge discovery across construction studies. Future construction robots will collaborate with field workers safely, improving productivity significantly and offsetting the growing labour shortage. The proposed research is essential to realizing this vision.