CS-GAN: Cross-Structure Generative Adversarial Networks for Chinese calligraphy translation
CS-GAN: Cross-Structure Generative Adversarial Networks for Chinese calligraphy translation
复制标题
DOI：
10.1016/j.knosys.2021.107334
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Yun Xiao;Wenlong Lei;Lei Lu;Xiaojun Chang;Xia Zheng;Xiaojiang Chen
中科院分区：
文献类型：
--
作者：
Yun Xiao;Wenlong Lei;Lei Lu;Xiaojun Chang;Xia Zheng;Xiaojiang Chen
摘要生成对抗网络（GANs）在跨域图像翻译方面取得了很大的进展。事实上，图像到图像的翻译任务经常会遇到两个域的结构差异，例如在不成对的中国书法数据集上的翻译。然而，现有的模型只能转换颜色和纹理特征，并保持结构不变（例如：在苹果到橙子的任务中，这些模型只转换苹果的颜色，但保持苹果的形状）。为了解决跨结构图像翻译问题，如中国书法的跨结构翻译问题，提出了一种新的生成对抗网络（Generative Adversarial Networks，GAN）模型CS-GAN。CS-GAN利用分布变换、参数化技巧和采样特征将S域的特征映射转换到T域。然后通过特征拼接生成域T的图像。本文在颜真卿、赵孟俯和欧阳询三位书法名家的三组具有结构差异的书法数据上对所提出的CS-GAN进行了验证。大量的实验结果表明，所提出的CS-GAN成功地转换了不同结构的中国书法数据，并优于最先进的模型。
Abstract Generative Adversarial Networks (GANs) have made great progress in cross-domain image translation. In fact, image-to-image translation tasks often encounter structural differences in two domains, such as translation on unpaired Chinese calligraphy dataset. However, existing models can only convert color and texture features and keep the structures unchanged (eg: in apples to oranges tasks, these models only convert the color of apples, but maintain the shape of apples). In order to address cross-structure image translation, such as cross-structure translation of Chinese calligraphy, a novel Generative Adversarial Networks (GAN) model, named CS-GAN, is proposed in this paper. In CS-GAN, distribution transform, reparameterization trick and sampling features are used to convert feature maps obtained from domain S to domain T. Then images of domain T are generated through features concatenation. The proposed CS-GAN is verified on three sets of Chinese calligraphic data with structural differences from three famous calligraphers, Yan Zhenqing, Zhao Mengfu and Ouyang Xun. The extensive experimental results show that the proposed CS-GAN successfully transforms the Chinese calligraphy data of different structures and outperforms the state of art models.