Semantic invariant cross-domain image generation with generative adversarial networks
Semantic invariant cross-domain image generation with generative adversarial networks
复制标题
DOI：
10.1016/j.neucom.2018.02.092
复制
发表时间：
2018-06
期刊：
影响因子：
6
通讯作者：
Xiaofeng Mao;Shuhui Wang;Liying Zheng;Qingming Huang
中科院分区：
文献类型：
--
作者：
Xiaofeng Mao;Shuhui Wang;Liying Zheng;Qingming Huang
最近，由于生成对抗网络中最先进的技术，许多工作在学习没有任何配对关系的输入图像和输出图像之间的映射方面取得了显着的性能。然而，传统的图像到图像翻译方法仅考虑视觉外观属性，无法在从源域到目标域的迁移学习过程中保持图像的真实语义。我们提出了一种新方法，利用 GAN 在域之间翻译不成对的图像，并保持高级语义抽象一致。我们的模型通过构建标签和注意力一致损失分别处理标签级别和空间级别的语义信息来控制图像的层次语义。在多个基准数据集上的实验结果表明，生成的样本在视觉上与目标图像相似，并且在语义上与其源对应图像一致。此外，实验还表明我们的方法可以有效提高无监督域适应问题的分类性能。
Recently, thanks to the state-of-the-art techniques in Generative Adversarial Networks, a lot of work achieves remarkable performance on learning the mapping between an input image and an output image without any paired relation. However, traditional methods on image-to-image translation merely consider the visual appearance properties, they fail to maintain the true semantics of an image during the transfer learning procedure from source to target domain. We propose a new approach that utilizes GAN to translate unpaired images between domains and remain high level semantic abstraction aligned. Our model controls the hierarchical semantics of images by processing semantic information on label level and spatial level respectively by constructing label and attention consistent losses. The experimental results on several benchmark datasets show that generated samples are both visually similar with target images and semantically consistent with their source counterparts. Furthermore, the experiment also suggests that our method can effectively improve the classification performance in unsupervised domain adaptation problem.