Improving semantic knowledge of urban environment based on data fusion and machine learning methods

该项目的第一部分将是了解复杂城市环境的数据融合。数据融合可以在处理流水线的各个阶段使用不同的数据并应用不同的方法来执行。由于这种多样性，至关重要的是要彻底了解数据融合中使用的方法和数据的优点和缺点，并选择最合适的进一步项目开发。在这部分工作中讨论的另一个要素是使用本体（基于语法，基于知识）及其在城市环境元素的语义描述中的潜力。 这项工作的目标之一是比较各种工具（商业，开源和基于编程）。因此，制定工作评估的措施或指标至关重要。在这项工作中使用的质量的决定因素是指使用数据融合在质量，数量或语义丰富方面改善结果。措施选择将基于文献和/或行业要求中的可用信息。这些指标可能涉及准确性、处理时间、成本降低、提取信息的范围（因为数据随其提供的信息而变化）以及应用部门。 为了比较本项目中研究的不同方法，我们的想法是使用数学模拟的特征（例如，表示墙的垂直平面）作为参考。接下来，从这些特征生成的点云将用于测试不同的工具（开源和商业）和分割方法（基于机器学习和深度学习解决方案）在重新创建/提取这些特征时的表现。这一步将使我了解参考数据集对于比较每个方法有多好。沿着点云处理，将使用几种方法来检测图像数据中的特征。这将深入了解来自各种传感器的数据如何代表一个特征，以及这些信息如何补充。 最后一步将是对实际数据（点云和图像）使用选定的方法（根据既定指标具有最佳性能）。数据将代表相同的感兴趣区域，以便能够融合通过数据处理收集的信息。结果应该提供语义上可及的信息，以丰富对城市环境（或所选对象类）的了解

The first part of the project will be about understanding data fusion for complex urban environments. Data fusion can be performed at various stages of the processing pipeline, using different data and applying different methods. Due to such diversity, it is crucial to thoroughly understand the advantages and disadvantages of the methods and data used in data fusion and to select the most appropriate for further project development. An additional element discussed in this part of the work is using ontologies (grammar-based, knowledge-based) and their potential in semantic description of the urban environment elements. One of this work's objectives is to compare various tools (commercial, open-source and programming based). Therefore, it is crucial to establish measures or indicators for work assessment. Determinants of quality used in this work refer to improving the results using data fusion in terms of quality, quantity, or semantic enrichment. The measures selection will be based on the information available in literature and/or industry requirements. The indicators may refer to accuracy performance, processing time, cost reduction, range of extracted information (as data vary with the information they deliver), and the application sector. To compare different methods investigated in this project, the idea is to use mathematically simulated features (e.g., a vertical plane representing a wall) as the reference. Next, a point cloud generated from these features will be used to test how different tools (open source and commercial) and segmentation methods (based on machine learning and deep learning solutions) perform in recreating/extracting these features. This step will allow me to understand how good the reference dataset is for comparing each method against. Along with point cloud processing, several approaches will be used to detect features in image data. This will give an insight into how well data from various sensors can represent a feature and how this information can complement. The final step will be to use selected methods (with best performance based on the established indicators) on real-world data (point cloud and images). Data will represent the same area of interest to allow the fusion of information collected through the data processing. The result should deliver semantically reach information to enrich knowledge of the city environment (or selected object classes)