Development and Evaluation of Real-like Synthetic Construction Images to Enhance the Performances of Deep Neural Network for Construction Applications

点击翻译按钮获取中文摘要

Deep neural networks (DNNs) are now seen as central to the future of artificial intelligence for construction robots. However, progress in robotics for the construction industry lags far behind that of other industries such as automobile and manufacturing. The problem-construction sites are extremely dynamic, rapidly changing, and crowded-poses an extreme challenge to the deployment of construction robots. DNN-powered AI is seen as the key to unlocking human-level understanding of construction sites. But DNN models need large sets of images to train on before they can obtain optimum performance. And these large training sets simply do not exist in the construction domain. Absent such training data, the outcome is a biased overfitted model with low accuracy and scalability. Against this backdrop, my short-term research goal is to improve the accuracy and scalability of DNN-based visual scene understanding models by deploying a novel training method that leverages non-real but real-looking virtual construction images. This short-term goal is an essential step in developing AI technology that achieves human-level visual scene understanding for construction contexts. It will be ground-breaking if we can automatically synthesize, generate, and label construction site images with no manual inputs, allowing DNNs to reach higher performance through a limitless quantity of training data, and allowing for the exploration of even deeper DNN architectures. There will be no legal issues in sharing non-real data; allowing such data to be used as the basis for fair competition and collaboration across research groups. To this end, my short-term research objectives are to: 1) Develop a computational framework that can automatically synthesize and label non-real but real-looking construction images that capture virtual construction worker avatars. 2) Build miniature-scale infrastructure that can record and label non-real but real-looking construction images that capture miniature-scale construction equipment. 3) Characterize the effect of the non-real images on a DNN's training, answering the following questions: (i) are the non-real construction images effective in improving the existing DNN models? and (ii) can a non-real construction image result in the same effect as a real image on DNN training? My short-term research outcome is to rapidly and efficiently expand training datasets of construction images by several orders of magnitude, with multiple important impacts on the study of DNNs in the construction domain, including: (i) dramatic improvements in DNN models for construction scene understanding; (ii) significant savings in time and resources; and (iii) enabling collective knowledge discovery across construction studies. Future construction robots will collaborate with field workers safely, improving productivity significantly and offsetting the growing labour shortage. The proposed research is essential to realizing this vision.