Synthesizing Optical and SAR Imagery From Land Cover Maps and Auxiliary Raster Data
Synthesizing Optical and SAR Imagery From Land Cover Maps and Auxiliary Raster Data
复制标题
DOI：
10.1109/tgrs.2021.3068532
复制
发表时间：
2020-11
影响因子：
8.2
通讯作者：
Gerald Baier;Antonin Deschemps;M. Schmitt;N. Yokoya
中科院分区：
文献类型：
--
作者：
Gerald Baier;Antonin Deschemps;M. Schmitt;N. Yokoya
我们利用生成对抗网络（gan）从土地覆盖图和辅助光栅数据合成光学RGB和合成孔径雷达（SAR）遥感图像。在遥感中，许多类型的数据，如数字高程模型（dem）或降水图，往往没有反映在土地覆盖图中，但仍然影响图像的内容或结构。在合成过程中包含这些数据可以提高生成图像的质量，并对其特征进行更多的控制。空间自适应归一化层融合两个输入，并应用于由编码器和解码器组成的成熟的生成器架构，以充分利用辅助光栅数据中的信息内容。当使用相应的数据集训练时，我们的方法成功地合成了中（10 m）和高（1 m）分辨率的图像。通过使用预训练的U-Net分割模型，我们展示了土地覆盖图和辅助信息的数据融合的优势，这些信息包括平均交联（miou）、像素精度和fr<s:1>起始距离（FIDs）。精心挑选的图像说明了融合信息如何避免合成图像中的歧义。通过稍微编辑输入，我们的方法可以用来合成现实的变化，即提高水位。源代码可在https://github.com/gbaier/rs_img_synth上获得，我们在https://ieee-dataport.org/open-access/geonrw上发布了新创建的高分辨率数据集。
We synthesize both optical RGB and synthetic aperture radar (SAR) remote sensing images from land cover maps and auxiliary raster data using generative adversarial networks (GANs). In remote sensing, many types of data, such as digital elevation models (DEMs) or precipitation maps, are often not reflected in land cover maps but still influence image content or structure. Including such data in the synthesis process increases the quality of the generated images and exerts more control on their characteristics. Spatially adaptive normalization layers fuse both inputs and are applied to a full-blown generator architecture consisting of encoder and decoder to take full advantage of the information content in the auxiliary raster data. Our method successfully synthesizes medium (10 m) and high (1 m) resolution images when trained with the corresponding data set. We show the advantage of data fusion of land cover maps and auxiliary information using mean intersection over unions (mIoUs), pixel accuracy, and Fréchet inception distances (FIDs) using pretrained U-Net segmentation models. Handpicked images exemplify how fusing information avoids ambiguities in the synthesized images. By slightly editing the input, our method can be used to synthesize realistic changes, i.e., raising the water levels. The source code is available at https://github.com/gbaier/rs_img_synth, and we published the newly created high-resolution data set at https://ieee-dataport.org/open-access/geonrw.