Semantic Segmentation in Geospatial Computer Vision

最近，由于深度学习的出现，计算机视觉取得了巨大的进步。在涉及视觉和空间处理的认知任务中，类似人类的表现已经实现。然而，这种高性能强烈依赖于数据集中训练样本的数量。为了解决这个问题，许多在线比赛都提供了大型基准数据集用于训练。创建图像或视频的训练数据集是一个昂贵且劳动密集型的过程。它需要大量的人员手动标记数据，并确保其正确性和完整性。一个数据集通常只能用于单个分类任务，这一事实加剧了这种情况。在使用遥感数据的地理空间领域尤其如此，其中大多数数据集用于建筑/非建筑分类。尽管在创建过程中存在困难，但研究人员依靠大型数据集来训练分类器，以帮助解决重建等更困难的问题。大规模城市区域的重建本质上是一个复杂的问题，涉及多个视觉任务。第一步是语义分割（*），其目标是将每个像素标记为城市特征类型，例如建筑物，道路，树木，植被，汽车，杂波等。接下来，根据像素的标签将其聚类成与其所代表的城市特征实例相对应的连续组。最后，对每个聚类进行重构，并根据聚类对应的城市特征类型应用自定义算法。因此，很明显，要实现完整的城市区域重建，必须首先解决与语义分割（*）相关的问题。该研究项目建立在我们最近在创建大规模逼真虚拟环境方面的研究成果之上，并专注于解决迄今为止确定的一些重大挑战。具体来说，这个DG将调查以下两个研究目标：1。少量的语义分割。目标是研究网络架构和训练范式，使网络训练只使用最小的训练示例集。2.可解释性。目的是研究分析和解释网络内部学习的方法，目的是在不需要进一步训练或微调的情况下，将大量预训练的语义分割网络重新用于与其主要任务相关的辅助任务。本研究计划可望为解决与电脑视觉领域高度实际相关的复杂问题作出重大贡献。（*）语义分割：每个*像素*都有自己的类标签；分类：*图像*有一个单一的类标签。用于描述博士生/候选人进展的缩写词：CE：综合考试，RP：研究提案，DS：博士研讨会

Recently there have been tremendous advances in computer vision due to the advent of deep learning. Human-like performance has already been achieved for cognitive tasks involving visual and spatial processing. However, this high performance is strongly dependent on the number of training examples in the dataset. To address this, many online competitions are offering large benchmark datasets for training. The creation of training datasets of images or video is a costly and labour-intensive process. It requires a significant number of people to label the data and ensure its correctness and completeness manually. This is exacerbated by the fact that a dataset can typically only be used for a single classification task. This is especially the case in the geospatial domain working with remote sensor data where most datasets are for building/non-building classification. Despite the difficulties involved in the creation, researchers rely on large datasets for training classifiers to assist in solving more difficult problems such as reconstruction. Reconstructing large-scale urban areas is an inherently complex problem that involves several vision tasks. The first step is semantic segmentation(*), where the objective is to label each pixel into an urban feature type, e.g., building, road, tree, vegetation, cars, clutter etc. Next, the pixels are clustered based on their labels into contiguous groups corresponding to instances of the urban features they represent. Finally, the reconstruction is performed on each cluster, where a customized algorithm is applied according to the urban feature type corresponding to the cluster. Hence, as it is evident, to achieve a complete urban-area reconstruction, one must first address the problems relating to semantic segmentation(*). This research program builds upon our most recent research outcomes in creating large-scale realistic virtual environments and focuses on addressing some of the significant challenges identified so far. Specifically, this DG will investigate the following two research objectives: 1.Few-shot semantic segmentation. The objective is to investigate network architectures and training paradigms which enable network training using only a minimal set of training examples. 2.Interpretability. The objective is to investigate methods for analyzing and interpreting what the network is learning internally with the goal of re-purposing the abundant pre-trained semantic segmentation networks on auxiliary tasks relating to their primary task, without further training or fine-tuning. This research program is expected to make substantial contributions to the solution of complex problems of high practical relevance to the field of computer vision. (*)Semantic segmentation: each *pixel* has its class label; Classification: the *image* has a single class label. Acronyms used to describe the progress of PhD students/candidates: CE: Comprehensive Exam RP: Research Proposal DS: Doctoral Seminar