Ground truth inference and quality control of geospatial data collection by paid crowdworkers for the efficient acquisition of training data for deep learning systems

目前，人们正在努力将卷积神经网络(CNN)等深度学习系统应用于遥感图像。然而，由于遥感图像的特殊性，标准的人工神经网络在遥感图像分析中的应用有限。从零开始训练专门的CNN是可取的，但由于缺乏所需的大量带注释的训练数据，这是不可能的。克劳德外包提供了一种提供此类数据的有效方法，这导致人们对使用这种方法从遥感图像中收集地理空间数据越来越感兴趣。然而，人群是由背景迥异的人组成的，他们中的大多数人都不熟悉地理空间数据采集标准。因此，我们必须期待质量非常不同的结果。该项目的目标是使受薪群众工作者能够从遥感图像中收集高质量的数据。该过程的设计使得即使没有参考数据也不需要对结果进行耗时的人工检查。我们提出了一种基于多数据采集的数据驱动方法来描述和提高采集数据的几何质量。首先，我们定义了一个综合质量度量，它用一个数值来量化地理对象的两个几何表示(一个是由众筹人员收集的，一个是对应的地面事实)的相似度。我们将使用信息论中的一种方法，根据统计评估得出这一衡量标准。作为下一步，我们将把多个表示集成到一个公共几何体中。我们将一方面使用质量度量来评估集成几何的质量，另一方面来优化集成过程。这即使在本质上也可以实现，而不需要与给定的地面事实进行比较。然后，我们想要调查使用CNN是否也可以在不收集多次数据的情况下实现自动质量评估。美国有线电视新闻网的输入将是一张遥感图像和一名众筹人员收集的个人几何图形。作为输出，美国有线电视新闻网将预测一项质量指标，该指标描述了物品收集的质量。使用这样的CNN，我们能够避免对同一对象进行多次收购的必要性。为了验证该方法的泛化能力，我们将其应用于具有不同特征的场景中。这需要我们的模型适应不同的领域。为此，我们使用了一种主动学习方法，该方法在众筹人员和CNN的相互作用中迭代地执行域适应。最后，我们讨论了可能的后续研究。

Currently, great efforts are being made to apply Deep Learning systems like Convolutional Neural Networks (CNN) also to remote sensing images. However, due to the peculiarities of remote sensing images, standard CNNs are of limited use for their analysis. It would be desirable to train specialized CNNs from scratch, but this is yet not possible due to the lack of the required large amount of annotated training data.Crowdsourcing offers an effective method for providing such data, which has led to increasing interest in using this method to collect geospatial data from remote sensing images. However, the crowd is composed of people with very different backgrounds, most of whom are not familiar with geospatial data collection standards. Therefore, we must expect results of very heterogeneous quality. The objective of this project is to enable the collection of high-quality data from remote sensing images by paid crowdworkers. The process is designed such that no time-consuming manual inspection of the results is necessary even if no reference data are available. We suggest a data-driven approach based on multiple data collection to describe and improve the geometric quality of the collected data. First, we define an integrated quality measure that quantifies the similarity of two geometric representations (one collected by a crowdworker, one the corresponding ground truth) of a geographic object with one numerical value. We will derive this measure based on statistical evaluations by using an approach from the information theory. As next step, we will integrate multiple representations into one common geometry. We will use the quality measure on the one hand to evaluate the quality of the integrated geometries and on the other hand to optimize the integration process. This can be realized even intrinsically without comparison to given ground truth.Then, we want to investigate if by using a CNN an automated quality evaluation can be realized also without multiple data collection. The input of this CNN will be a remote sensing image and one individual geometry collected by a crowdworker. As output, the CNN shall predict a quality measure that describes how good the object was collected. Using such a CNN, we are able to avoid the necessity of multiple acquisitions of the same object. Consequently, collecting data will be much cheaper.In order to validate the generalizability of our approach, we apply it to scenes of quite different characteristics. This requires an adaptation of our model to different domains. For this purpose, we use an Active Learning approach, which iteratively performs domain adaptation in the interplay of crowdworkers and a CNN. Finally, we address possible follow-up research.