Building a Parallel Universe Image Synthesis from Land Cover Maps and Auxiliary Raster Data
Building a Parallel Universe Image Synthesis from Land Cover Maps and Auxiliary Raster Data
复制标题
DOI：
--
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Gerald Baier;Antonin Deschemps;M. Schmitt;N. Yokoya
中科院分区：
文献类型：
--
作者：
Gerald Baier;Antonin Deschemps;M. Schmitt;N. Yokoya
我们利用gan合成了陆地覆盖图和辅助栅格数据的光学RGB和SAR遥感图像。在遥感中，许多类型的数据，如数字高程模型或降水图，往往没有反映在土地覆盖图中，但仍然影响图像的内容或结构。在合成过程中包含这些数据可以提高生成图像的质量，并对其特征进行更多的控制。我们的方法通过空间自适应归一化层融合两个输入，之前发表为SPADE语义图像合成。与SPADE相比，这些归一化层应用于由编码器和解码器组成的成熟的生成器架构，以充分利用辅助光栅数据中的信息内容。当使用相应的数据集进行训练时，我们的方法成功地合成了中（10m）和高（1m）分辨率的图像。我们展示了使用平均交叉点的土地覆盖图和辅助信息的数据融合优于使用预训练的U-Net分割模型的联合、像素精度和FID。精心挑选的图像说明了融合信息如何避免合成图像中的歧义。通过稍微编辑输入，我们的方法可以用来合成现实的变化，即提高水位。源代码可在https://github.com/gbaier/rs_img_synth上获得，我们在https://ieee-dataport.org/open-access/geonrw上发布了新创建的高分辨率数据集。
We synthesize both optical RGB and SAR remote sensing images from land cover maps and auxiliary raster data using GANs. In remote sensing many types of data, such as digital elevation models or precipitation maps, are often not reflected in land cover maps but still influence image content or structure. Including such data in the synthesis process increases the quality of the generated images and exerts more control on their characteristics. Our method fuses both inputs by spatially adaptive normalization layers, previously published as SPADE semantic image synthesis. In contrast to SPADE, these normalization layers are applied to a full-blown generator architecture consisting of encoder and decoder, to take full advantage of the information content in the auxiliary raster data. Our method successfully synthesizes medium (10m) and high (1m) resolution images, when trained with the corresponding dataset. We show the advantage of data fusion of land cover maps and auxiliary information using mean intersection over union, pixel accuracy and FID using pre-trained U-Net segmentation models. Handpicked images exemplify how fusing information avoids ambiguities in the synthesized images. By slightly editing the input our method can be used to synthesize realistic changes, i.e., raising the water levels. The source code is available at https://github.com/gbaier/rs_img_synth and we published the newly created high-resolution dataset at https://ieee-dataport.org/open-access/geonrw.