TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images
TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images
复制标题
DOI：
10.1007/978-3-030-58548-8_2
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Jianxin Lin;Yingxue Pang;Yingce Xia;Zhibo Chen;Jiebo Luo
中科院分区：
文献类型：
--
作者：
Jianxin Lin;Yingxue Pang;Yingce Xia;Zhibo Chen;Jiebo Luo
无监督图像到图像翻译（UI2I）任务处理学习两个没有配对图像的域之间的映射。虽然现有的UI2I方法通常需要来自不同领域的大量未配对图像进行训练，但在许多情况下，训练数据非常有限。在本文中，我们认为，即使每个域包含一个单一的图像，UI2I仍然可以实现。为此，我们提出了TuiGAN，这是一种生成模型，只在两个未配对的图像上训练，相当于一次性无监督学习。使用TuiGAN，图像以粗到细的方式进行翻译，其中生成的图像从全局结构逐渐细化到局部细节。我们进行了大量的实验，以验证我们的通用方法可以在各种各样的UI2I任务上优于强基线。此外，TuiGAN能够实现与使用足够数据训练的最先进的UI2I模型相当的性能。
An unsupervised image-to-image translation (UI2I) task deals with learning a mapping between two domains without paired images. While existing UI2I methods usually require numerous unpaired images from different domains for training, there are many scenarios where training data is quite limited. In this paper, we argue that even if each domain contains a single image, UI2I can still be achieved. To this end, we propose TuiGAN, a generative model that is trained on only two unpaired images and amounts to one-shot unsupervised learning. With TuiGAN, an image is translated in a coarse-to-fine manner where the generated image is gradually refined from global structures to local details. We conduct extensive experiments to verify that our versatile method can outperform strong baselines on a wide variety of UI2I tasks. Moreover, TuiGAN is capable of achieving comparable performance with the state-of-the-art UI2I models trained with sufficient data.