Automatic Creation and Real-time Update of Detailed 3D Maps Using LiDAR and Images

详细和最新的3D地图是智能城市和智能交通系统等应用的重要组成部分。近年来，对这种3D地图的需求急剧增加。这些需求尤其来自自动驾驶汽车和自动物流。根据Research and Markets的数据，在2017-2027年预测期内，全球自动驾驶汽车市场收入预计将以39.6%的复合年增长率（复合年增长率）增长，到2027年将达到1268亿美元。详细和修订的3D地图是自主应用的先决条件，该提案解决了与自动绘图相关的基本问题。拟议的三个项目存在于一个统一的框架下，该框架集成了多模式数据，用于自动创建详细的3D地图。由于深度学习在对象识别和分类方面的巨大成功，从图像中识别和分类对象已经在很大程度上实现。然而，这对于缺乏有效的3D表示的3D点云是不正确的。第一个项目旨在开发一个3D深度学习系统，该系统使用不同的数据结构来表示点云，用于3D点云的语义分割。第二个项目涉及开发一个使用数字表面模型信息的卫星图像分析系统。可以使用LiDAR、图像或其组合来生成3D地图。两者的结合构成了一种传感器融合方法，该方法利用各自的优点来克服其局限性。给定高分辨率卫星图像和相应的DSM，可以被视为3D点云，该系统同时分割和标记图像的高精度。最后一个项目涉及使用移动的LiDAR和来自航空图像的点云进行细粒度立面建模。尽管详细的3D建筑物模型是3D地图的关键组成部分之一，但它们尚未得到充分的研究。我们的目标是开发一个适用于各种风格的细粒度立面建模系统。拟议的研究建立在我以前的工作基础上，将大大推动目标领域的前沿研究。研究结果将对地图和导航行业、基于位置的服务提供商以及整个加拿大的社会效益产生重大的经济影响。

Detailed and up-to-date 3D maps are essential components of applications, such as smart cities and intelligent transportation systems. In recent years, the demand for such 3D maps has dramatically increased. These needs arise particularly from self-driving cars and autonomous logistics. According to Research and Markets, the global autonomous vehicles market revenue is expected to grow at a CAGR (compound annual growth rate) of 39.6% during the forecast period 2017-2027, reaching $126.8 billion by 2027. Detailed and revised 3D maps are a prerequisite for autonomous applications, and this proposal addresses the fundamental issues associated with automated mapping. The proposed three projects exist under a unified framework that integrates multimodal data for the automatic creation of detailed 3D maps. Due to the great success of deep learning in object recognition and classification, the recognition and classification of objects from images has been largely achieved. However, this is not true for 3D point clouds, which suffer from a lack of effective 3D representation. The first project aims to develop a 3D deep learning system that uses different data structures to represent point clouds for the semantic segmentation of 3D point clouds. The second project involves the development of a satellite imagery parsing system that uses digital surface model (DSM) information. 3D maps can be generated using LiDAR, images, or a combination thereof. The combination of both constitutes a sensor fusion approach that uses the advantages of each to overcome their limitations. Given high-resolution satellite images and the corresponding DSMs, which can be regarded as 3D point clouds, the system simultaneously segments and labels images with high precision. The last project involves fine-grained façade modeling using mobile LiDAR and point clouds from aerial images. Despite detailed 3D building models being one of the key components in 3D maps, they have not been sufficiently researched. We aim to develop a fine-grained façade modeling system that is applicable for various styles. The proposed research builds upon my previous work and will significantly drive cutting-edge research in the targeted areas. The results will have a substantial economic impact on the mapping and navigation industry, location-based service providers, as well as social benefits to Canada as a whole.