Automatic Creation and Real-time Update of Detailed 3D Maps Using LiDAR and Images

详细和最新的3D地图是智能城市和智能交通系统等应用程序的基本组件。近年来，对这类3D地图的需求急剧增加。这些需求尤其来自自动驾驶汽车和自动物流。根据Research and Markets的预测，在20172027预测期内，全球自动驾驶汽车市场的收入预计将以39.6%的复合年增长率增长，到2027年将达到1268亿美元。详细和修订的3D地图是自主应用的先决条件，本提案解决了与自动化地图相关的基本问题。拟议的三个项目是在一个统一的框架下进行的，该框架集成了多模式数据以自动创建详细的3D地图。*由于深度学习在目标识别和分类方面取得了巨大成功，已基本实现了对图像中目标的识别和分类。然而，这并不适用于3D点云，因为它缺乏有效的3D表示。第一个项目旨在开发一个3D深度学习系统，该系统使用不同的数据结构来表示点云，用于3D点云的语义分割。第二个项目涉及开发使用数字地面模型信息的卫星图像解析系统。可以使用激光雷达、图像或其组合来生成3D地图。两者的结合构成了一种传感器融合方法，它利用各自的优势来克服各自的局限性。在给定高分辨率卫星图像和相应的DSM(可视为三维点云)的情况下，系统同时对图像进行高精度的分割和标记。最后一个项目涉及使用移动激光雷达和来自航拍图像的点云进行细粒度立面建模。尽管详细的3D建筑物模型是3D地图的关键组成部分之一，但它们还没有得到充分的研究。我们的目标是开发一个适用于各种风格的细粒度外墙建模系统。拟议的研究建立在我以前工作的基础上，并将极大地推动目标领域的尖端研究。这一结果将对地图和导航行业、基于位置的服务提供商以及整个加拿大的社会效益产生重大的经济影响。

Detailed and up-to-date 3D maps are essential components of applications, such as smart cities and intelligent transportation systems. In recent years, the demand for such 3D maps has dramatically increased. These needs arise particularly from self-driving cars and autonomous logistics. According to Research and Markets, the global autonomous vehicles market revenue is expected to grow at a CAGR (compound annual growth rate) of 39.6% during the forecast period 20172027, reaching $126.8 billion by 2027. Detailed and revised 3D maps are a prerequisite for autonomous applications, and this proposal addresses the fundamental issues associated with automated mapping. The proposed three projects exist under a unified framework that integrates multimodal data for the automatic creation of detailed 3D maps.******Due to the great success of deep learning in object recognition and classification, the recognition and classification of objects from images has been largely achieved. However, this is not true for 3D point clouds, which suffer from a lack of effective 3D representation. The first project aims to develop a 3D deep learning system that uses different data structures to represent point clouds for the semantic segmentation of 3D point clouds. The second project involves the development of a satellite imagery parsing system that uses digital surface model (DSM) information. 3D maps can be generated using LiDAR, images, or a combination thereof. The combination of both constitutes a sensor fusion approach that uses the advantages of each to overcome their limitations. Given high-resolution satellite images and the corresponding DSMs, which can be regarded as 3D point clouds, the system simultaneously segments and labels images with high precision. The last project involves fine-grained façade modeling using mobile LiDAR and point clouds from aerial images. Despite detailed 3D building models being one of the key components in 3D maps, they have not been sufficiently researched. We aim to develop a fine-grained façade modeling system that is applicable for various styles. The proposed research builds upon my previous work and will significantly drive cutting-edge research in the targeted areas. The results will have a substantial economic impact on the mapping and navigation industry, location-based service providers, as well as social benefits to Canada as a whole.*****************