BigEarthNet Deep Learning Models with A New Class-Nomenclature for Remote Sensing Image Understanding
BigEarthNet Deep Learning Models with A New Class-Nomenclature for Remote Sensing Image Understanding
复制标题
DOI：
--
复制
发表时间：
2020-01
期刊：
影响因子：
--
通讯作者：
Gencer Sumbul;Jian Kang;Tristan Kreuziger;F. Marcelino;H. Costa;P. Benevides;M. Caetano;B. Demir
中科院分区：
文献类型：
--
作者：
Gencer Sumbul;Jian Kang;Tristan Kreuziger;F. Marcelino;H. Costa;P. Benevides;M. Caetano;B. Demir
深度神经网络在遥感 (RS) 图像分析框架中的成功取决于大量注释图像的可用性。 BigEarthNet 是一个新的大规模 Sentinel-2 基准档案，最近在 RS 中引入，以推进深度学习 (DL) 研究。 BigEarthNet 中的每个图像块都根据 2018 年 CORINE 土地覆盖 (CLC) 地图基于其最主题详细的 3 级类别命名法提供的多标签进行注释。 BigEarthNet 使需要大量数据的深度学习算法能够在多标签 RS 图像检索和分类的背景下达到高性能。然而，初步研究表明，仅考虑（单一日期）Sentinel-2 图像来准确描述某些 CLC 类别具有挑战性。为了进一步提高 BigEarthNet 的有效性，在本文中，我们引入了一种替代类命名法，使 DL 模型能够更好地学习和描述 Sentinel-2 图像的复杂空间和光谱信息内容。这是通过根据 Sentinel-2 图像的属性在 19 类新命名法中解释和排列 CLC Level-3 命名法来实现的。然后，BigEarthNet 的新类命名法在最先进的 DL 模型（即 16 和 19 层深度的 VGG 模型 [VGG16 和 VGG19] 和 50、101 和 152 层深度的 ResNet 模型 [ResNet50、ResNet101、ResNet152] 以及 K-Branch CNN 模型）中使用多标签分类。实验结果表明，在 BigEarthNet 上从头开始训练的模型优于在 ImageNet 上预训练的模型，特别是在涉及农业和其他植被和自然环境等复杂类别时。所有深度学习模型都是公开的，为指导 RS 中基于内容的图像检索和场景分类问题的未来进展提供了重要的资源。
Success of deep neural networks in the framework of remote sensing (RS) image analysis depends on the availability of a high number of annotated images. BigEarthNet is a new large-scale Sentinel-2 benchmark archive that has been recently introduced in RS to advance deep learning (DL) studies. Each image patch in BigEarthNet is annotated with multi-labels provided by the CORINE Land Cover (CLC) map of 2018 based on its most thematic detailed Level-3 class nomenclature. BigEarthNet has enabled data-hungry DL algorithms to reach high performance in the context of multi-label RS image retrieval and classification. However, initial research demonstrates that some CLC classes are challenging to be accurately described by considering only (single-date) Sentinel-2 images. To further increase the effectiveness of BigEarthNet, in this paper we introduce an alternative class-nomenclature to allow DL models for better learning and describing the complex spatial and spectral information content of the Sentinel-2 images. This is achieved by interpreting and arranging the CLC Level-3 nomenclature based on the properties of Sentinel-2 images in a new nomenclature of 19 classes. Then, the new class-nomenclature of BigEarthNet is used within state-of-the-art DL models (namely VGG model at the depth of 16 and 19 layers [VGG16 and VGG19] and ResNet model at the depth of 50, 101 and 152 layers [ResNet50, ResNet101, ResNet152] as well as K-Branch CNN model) in the context of multi-label classification. Experimental results show that the models trained from scratch on BigEarthNet outperform those pre-trained on ImageNet, especially in relation to some complex classes including agriculture and other vegetated and natural environments. All DL models are made publicly available, offering an important resource to guide future progress on content based image retrieval and scene classification problems in RS.