Semantic Segmentation in Geospatial Computer Vision

地理空间计算机视觉中的语义分割

• 批准号: RGPIN-2021-03479
• 负责人: Poullis, Charalambos
• 金额: $ 2.11万
• 依托单位: Concordia University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750121
• 关键词: Semantic; Segmentation; Geospatial; Computer; Vision

Recently there have been tremendous advances in computer vision due to the advent of deep learning. Human-like performance has already been achieved for cognitive tasks involving visual and spatial processing. However, this high performance is strongly dependent on the number of training examples in the dataset. To address this, many online competitions are offering large benchmark datasets for training. The creation of training datasets of images or video is a costly and labour-intensive process. It requires a significant number of people to label the data and ensure its correctness and completeness manually. This is exacerbated by the fact that a dataset can typically only be used for a single classification task. This is especially the case in the geospatial domain working with remote sensor data where most datasets are for building/non-building classification. Despite the difficulties involved in the creation, researchers rely on large datasets for training classifiers to assist in solving more difficult problems such as reconstruction. Reconstructing large-scale urban areas is an inherently complex problem that involves several vision tasks. The first step is semantic segmentation(*), where the objective is to label each pixel into an urban feature type, e.g., building, road, tree, vegetation, cars, clutter etc. Next, the pixels are clustered based on their labels into contiguous groups corresponding to instances of the urban features they represent. Finally, the reconstruction is performed on each cluster, where a customized algorithm is applied according to the urban feature type corresponding to the cluster. Hence, as it is evident, to achieve a complete urban-area reconstruction, one must first address the problems relating to semantic segmentation(*). This research program builds upon our most recent research outcomes in creating large-scale realistic virtual environments and focuses on addressing some of the significant challenges identified so far. Specifically, this DG will investigate the following two research objectives: 1.Few-shot semantic segmentation. The objective is to investigate network architectures and training paradigms which enable network training using only a minimal set of training examples. 2.Interpretability. The objective is to investigate methods for analyzing and interpreting what the network is learning internally with the goal of re-purposing the abundant pre-trained semantic segmentation networks on auxiliary tasks relating to their primary task, without further training or fine-tuning. This research program is expected to make substantial contributions to the solution of complex problems of high practical relevance to the field of computer vision. (*)Semantic segmentation: each *pixel* has its class label; Classification: the *image* has a single class label. Acronyms used to describe the progress of PhD students/candidates: CE: Comprehensive Exam RP: Research Proposal DS: Doctoral Seminar

最近，由于深度学习的出现，计算机视觉取得了巨大的进步。在涉及视觉和空间处理的认知任务中，已经实现了类似人类的表现。然而，这种高性能在很大程度上取决于数据集中训练样本的数量。为了解决这个问题，许多在线竞赛都提供了大型基准数据集进行训练。创建图像或视频的训练数据集是一个昂贵且劳动密集型的过程。它需要大量的人来标记数据，并手动确保其正确性和完整性。数据集通常只能用于单个分类任务的事实加剧了这一点。在使用遥感器数据的地理空间领域尤其如此，因为大多数数据集用于建筑物/非建筑物分类。尽管创建过程中存在困难，但研究人员仍然依赖大型数据集来训练分类器，以帮助解决更困难的问题，例如重建。重建大规模的城市地区是一个固有的复杂问题，涉及到几个视觉任务。第一步是语义分割（*），目标是将每个像素标记为城市特征类型，例如，建筑物、道路、树木、植被、汽车、杂物等。接下来，基于像素的标签将像素聚类到与它们所表示的城市特征的实例相对应的连续组中。最后，对每个聚类进行重建，其中根据聚类对应的城市特征类型应用定制的算法。因此，很明显，要实现完整的城市地区重建，必须首先解决与语义分割（*）有关的问题。该研究计划建立在我们最近在创建大规模逼真虚拟环境方面的研究成果的基础上，并专注于解决迄今为止发现的一些重大挑战。具体而言，本研究将探讨以下两个研究目标：1。少镜头语义分割。我们的目标是调查网络架构和训练范式，使网络训练只使用一个最小的训练样本集。2.Interpretability.我们的目标是研究分析和解释网络内部学习内容的方法，目的是在与其主要任务相关的辅助任务上重新利用丰富的预训练语义分割网络，而无需进一步训练或微调。该研究计划预计将为解决与计算机视觉领域具有高度实际相关性的复杂问题做出实质性贡献。（*）语义分割：每个 * 像素 * 都有它的类标签;分类：* 图像 * 只有一个类标签。用于描述博士生/候选人进展的首字母缩略词：CE：综合考试RP：研究提案DS：博士研讨会