Ground truth inference and quality control of geospatial data collection by paid crowdworkers for the efficient acquisition of training data for deep learning systems

目前，人们正在努力将卷积神经网络（CNN）等深度学习系统应用于遥感图像。然而，由于遥感图像的特殊性，标准cnn对遥感图像的分析作用有限。从头开始训练专门的cnn是可取的，但由于缺乏所需的大量带注释的训练数据，这还不可能实现。众包为提供这类数据提供了一种有效的方法，这使得人们越来越有兴趣使用这种方法从遥感图像中收集地理空间数据。然而，这群人是由背景非常不同的人组成的，他们中的大多数人都不熟悉地理空间数据收集标准。因此，我们必须期望得到质量非常不均匀的结果。该项目的目标是通过付费众包工作者从遥感图像中收集高质量数据。该过程的设计使得即使没有可用的参考数据，也不需要对结果进行耗时的人工检查。我们提出了一种基于多个数据收集的数据驱动方法来描述和提高所收集数据的几何质量。首先，我们定义了一个综合质量度量，该度量量化具有一个数值的地理对象的两个几何表示（一个由众包工作者收集，一个对应的地面真值）的相似性。我们将利用信息论的方法，在统计评估的基础上推导出这一度量。下一步，我们将把多个表示集成到一个公共几何中。我们将使用质量度量一方面来评估集成几何的质量，另一方面来优化集成过程。这甚至可以在本质上实现，而无需与给定的基础真理进行比较。然后，我们想要研究使用CNN是否可以在没有多次数据收集的情况下实现自动质量评估。这个CNN的输入将是一幅遥感图像和一个由众包工作者收集的单个几何图形。作为输出，CNN应该预测一个质量度量，描述对象被收集得有多好。使用这样的CNN，我们能够避免对同一对象进行多次获取的必要性。因此，收集数据将会便宜得多。为了验证我们方法的泛化性，我们将其应用于具有完全不同特征的场景。这需要我们的模型适应不同的领域。为此，我们使用了一种主动学习方法，该方法在众工和CNN的相互作用中迭代地执行域适应。最后，我们讨论了可能的后续研究。

Currently, great efforts are being made to apply Deep Learning systems like Convolutional Neural Networks (CNN) also to remote sensing images. However, due to the peculiarities of remote sensing images, standard CNNs are of limited use for their analysis. It would be desirable to train specialized CNNs from scratch, but this is yet not possible due to the lack of the required large amount of annotated training data.Crowdsourcing offers an effective method for providing such data, which has led to increasing interest in using this method to collect geospatial data from remote sensing images. However, the crowd is composed of people with very different backgrounds, most of whom are not familiar with geospatial data collection standards. Therefore, we must expect results of very heterogeneous quality. The objective of this project is to enable the collection of high-quality data from remote sensing images by paid crowdworkers. The process is designed such that no time-consuming manual inspection of the results is necessary even if no reference data are available. We suggest a data-driven approach based on multiple data collection to describe and improve the geometric quality of the collected data. First, we define an integrated quality measure that quantifies the similarity of two geometric representations (one collected by a crowdworker, one the corresponding ground truth) of a geographic object with one numerical value. We will derive this measure based on statistical evaluations by using an approach from the information theory. As next step, we will integrate multiple representations into one common geometry. We will use the quality measure on the one hand to evaluate the quality of the integrated geometries and on the other hand to optimize the integration process. This can be realized even intrinsically without comparison to given ground truth.Then, we want to investigate if by using a CNN an automated quality evaluation can be realized also without multiple data collection. The input of this CNN will be a remote sensing image and one individual geometry collected by a crowdworker. As output, the CNN shall predict a quality measure that describes how good the object was collected. Using such a CNN, we are able to avoid the necessity of multiple acquisitions of the same object. Consequently, collecting data will be much cheaper.In order to validate the generalizability of our approach, we apply it to scenes of quite different characteristics. This requires an adaptation of our model to different domains. For this purpose, we use an Active Learning approach, which iteratively performs domain adaptation in the interplay of crowdworkers and a CNN. Finally, we address possible follow-up research.