Generating and Maintaining Multi-Level 3D City Models Using Advanced Multi-Modal Image Processing

以三维几何形式呈现城市物体和结构（如建筑物）的3D城市模型正越来越多地用于广泛的应用，包括智能城市、城市规划、灾害管理和安全。三维城市模型生成在不同的细节水平（LoD）；例如，LoD1只包含建筑物的粗略块表示，而LoD3包含精致的建筑结构以及建筑物内门窗的位置。LoD=3模型处理建筑物和城市物体的外部，而LoD4侧重于建筑内部。不同的应用程序需要不同层次的细节。因此，有必要对三维城市模型进行多层次的表示，以适应各种需求。虽然三维城市模型在不同的测绘学和城市管理应用中至关重要，但制作高LoDs（=2）的模型仍然是昂贵和劳动密集型的，因此，这类模型的城市发展速度被延长。因此，世界上只有少数城市拥有有限的多层次3D城市模型（主要是LoD2）。此外，目前维护3D城市模型非常具有挑战性，因此在许多应用中，3D城市模型不是更新，而是在一段时间后重新生成。我的研究项目的长期目标是开发突破性的方法来自动生成和维护符合标准的多层次3D城市模型，使用摄影测量的基础上结合图像处理中的学习方法。本研究计划的短期目标是开发自动化方法：(1)生成3D城市模型，(2)检测现有模型的变化，(3)更新模型以保持LoD=3城市模型（重点是建筑外观）。我和我的研究生将结合现代学习技术，即机器学习和深度学习，以及基本的摄影测量概念，进一步发展我们在变化检测和传感器建模方面所取得的成就，以实现本研究计划的短期目标。为此，我们将使用从不同角度（如倾斜和最低点）、不同平台（如地面和空中）和各种传感器（如多光谱、激光雷达）拍摄的图像，这些图像被称为多模态图像。为了实现短期目标，我将在新不伦瑞克大学大地测量与信息工程系招聘和培训9名遥感，摄影测量和地理信息系统领域的HQP（2名博士，2名硕士，5名学士）。这一举措将使加拿大成为该领域的领先国家之一，使加拿大人能够从三维城市模型在灾害管理和智能城市等不同应用中提供的机会中受益。

3D city models that present urban objects and structures, such as buildings, in a three-dimensional geometry are being increasingly used for a wide range of applications, including smart cities, urban planning, disaster management and security. 3D city models are generated in different Levels of Detail (LoD); for example, LoD1 only contains a rough block representation of buildings while LoD3 contains the delicate architectural structures as well as the location of the windows and doors within buildings. The LoD=3 models deal with the exterior of buildings and urban objects, while LoD4 focuses on building interiors. Different applications require different levels of detail. Thus, it is necessary to have multi-level representations of 3D city models to accommodate various requirements. Although 3D city models are vital in different Geomatics and Urban Management applications, producing the models with high LoDs (=2) is still costly and labor-intensive - thus, the development rate of such models for cities is prolonged. Consequently, only a few cities in the world have limited multi-level 3D city models (mainly up to LoD2). Furthermore, maintaining 3D city models is currently very challenging, hence in many applications, instead of updating, 3D city models are regenerated after a certain period. The long term objective of my research program is to develop breakthrough methods to automatically generate and maintain standards-compliant multi-level 3D city models, using a photogrammetric basis combined by learning methods in image processing. The short term objectives of this research program are to develop automatic methods to (1) generate 3D city models, (2) detect changes in the existing models and (3) update the models to maintain the LoD=3 city models (focusing on building exteriors). My grad students and I will combine modern learning techniques, i.e. machine learning and deep learning, and basic photogrammetric concepts to further develop what we have achieved up to date in terms of change detection and sensor modeling to pursue the short goals of this research program. For this purpose, we will use images taken from different angles (e.g. oblique and nadir), different platforms (e.g. terrestrial and airborne) and various sensors (e.g. multispectral, LiDAR) that are referred to as multi-modal images. To pursue the short term objectives, I am going to hire and train 9 HQP in total (2 PhD, 2 MSc, 5 BSc) in the field of Remote Sensing, Photogrammetry and GIS working at the Department of Geodesy and Geomatics Engineering, University of New Brunswick. This initiative will place Canada within the group of leading countries in this field, enabling Canadians to benefit from the opportunities provided by 3D city models in different applications such as disaster management and smart cities.