Semantic Segmentation in Geospatial Computer Vision

最近，由于深度学习的出现，计算机视觉取得了巨大的进步。在涉及视觉和空间处理的认知任务中，已经取得了与人类相似的表现。然而，这种高性能强烈地依赖于数据集中的训练样本的数量。为了解决这一问题，许多在线比赛都提供了大型基准数据集用于培训。创建图像或视频的训练数据集是一个昂贵且劳动密集型的过程。它需要大量的人来手动标记数据并确保其正确性和完整性。数据集通常只能用于单个分类任务，这一事实加剧了这种情况。在处理遥感数据的地理空间领域尤其如此，其中大多数数据集用于建筑物/非建筑物分类。尽管创建过程中涉及到困难，但研究人员依赖大数据集来训练分类器，以帮助解决更困难的问题，如重建。城市大规模改造是一个内在的复杂问题，涉及多个视觉任务。第一步是语义分割(*)，其中目标是将每个像素标记为城市特征类型，例如建筑、道路、树木、植被、汽车、杂乱等。接下来，基于其标签将像素聚类成对应于它们所代表的城市特征的实例的邻接组。最后，对每个簇进行重建，根据簇对应的城市特征类型应用定制的算法。因此，很明显，要实现完整的市区重建，必须首先解决与语义分割相关的问题(*)。这项研究计划建立在我们在创建大规模逼真虚拟环境方面的最新研究成果的基础上，并专注于解决到目前为止确定的一些重大挑战。具体而言，本文将研究以下两个方面的研究目标：1.镜头语义切分。目标是研究网络体系结构和培训范例，使网络培训仅使用最少的一组培训示例。2.可解释性。其目的是研究分析和解释网络内部学习内容的方法，目的是在不进一步训练或微调的情况下，将丰富的预先训练的语义分割网络用于与其主要任务相关的辅助任务。这一研究计划有望为解决计算机视觉领域具有高度实用意义的复杂问题做出实质性贡献。(*)语义分割：每个*像素*都有分类标签；分类：*图像*只有一个分类标签。用于描述博士生/候选人进展的首字母缩写：CE：综合考试RP：研究建议DS：博士研讨会

Recently there have been tremendous advances in computer vision due to the advent of deep learning. Human-like performance has already been achieved for cognitive tasks involving visual and spatial processing. However, this high performance is strongly dependent on the number of training examples in the dataset. To address this, many online competitions are offering large benchmark datasets for training. The creation of training datasets of images or video is a costly and labour-intensive process. It requires a significant number of people to label the data and ensure its correctness and completeness manually. This is exacerbated by the fact that a dataset can typically only be used for a single classification task. This is especially the case in the geospatial domain working with remote sensor data where most datasets are for building/non-building classification. Despite the difficulties involved in the creation, researchers rely on large datasets for training classifiers to assist in solving more difficult problems such as reconstruction. Reconstructing large-scale urban areas is an inherently complex problem that involves several vision tasks. The first step is semantic segmentation(*), where the objective is to label each pixel into an urban feature type, e.g., building, road, tree, vegetation, cars, clutter etc. Next, the pixels are clustered based on their labels into contiguous groups corresponding to instances of the urban features they represent. Finally, the reconstruction is performed on each cluster, where a customized algorithm is applied according to the urban feature type corresponding to the cluster. Hence, as it is evident, to achieve a complete urban-area reconstruction, one must first address the problems relating to semantic segmentation(*). This research program builds upon our most recent research outcomes in creating large-scale realistic virtual environments and focuses on addressing some of the significant challenges identified so far. Specifically, this DG will investigate the following two research objectives: 1.Few-shot semantic segmentation. The objective is to investigate network architectures and training paradigms which enable network training using only a minimal set of training examples. 2.Interpretability. The objective is to investigate methods for analyzing and interpreting what the network is learning internally with the goal of re-purposing the abundant pre-trained semantic segmentation networks on auxiliary tasks relating to their primary task, without further training or fine-tuning. This research program is expected to make substantial contributions to the solution of complex problems of high practical relevance to the field of computer vision. (*)Semantic segmentation: each *pixel* has its class label; Classification: the *image* has a single class label. Acronyms used to describe the progress of PhD students/candidates: CE: Comprehensive Exam RP: Research Proposal DS: Doctoral Seminar