Contour-enhanced CycleGAN framework for style transfer from scenery photos to Chinese landscape paintings
Contour-enhanced CycleGAN framework for style transfer from scenery photos to Chinese landscape paintings
复制标题
DOI：
10.1007/s00521-022-07432-w
复制
发表时间：
2022-06
影响因子：
6
通讯作者：
Xianlin Peng;Shenglin Peng;Qiyao Hu;Jinye Peng;Jiaxin Wang;Xinyu Liu;Jianping Fan
中科院分区：
文献类型：
--
作者：
Xianlin Peng;Shenglin Peng;Qiyao Hu;Jinye Peng;Jiaxin Wang;Xinyu Liu;Jianping Fan
基于生成对抗网络模型的图像风格迁移已成为一个重要的研究领域。在这些生成对抗网络模型中，CycleGAN 的一个明显优势是，当数据不配对时，它可以在多个域之间传输。为了近似具有中国画特色的纹理方法——“寸法”的效果，本文提出了一种图像风格迁移框架，以实现从风景照片到中国山水画的迁移。我们设计了一个轮廓增强翻译分支，它可以通过计算梯度图的边缘检测算子有效地引导从照片到绘画的转换。仿真结果表明，该方法可以将真实风景照片转换为中国山水画。 Inception Score表明轮廓增强可以使生成集在对图像边缘的敏感度上表现更好。生成图像与“Cun法”数据集之间的Kernel Inception距离和Inception-based Structural Comparison表明，轮廓增强可以使生成图像更接近“Cun法”效果。与Kernel Inception距离和Frechet-Inception距离相比，本文提出的基于Inception的结构相似度直接关注相似度，我们的模型生成的图像的平均特征与“Cun方法”集之间的相似度为97.89％，复合相似度度量为0.92。该方法在神经图像评估指标下的表现也优于 MUNIT、NiceGAN、CycleGAN 和 U-GAT-IT 参考模型。这表明边缘算子的引入使得生成的风景画更加美观，尤其是在风景照片边缘信息丰富的情况下。
Image style transfer based on the generative adversarial network model has become an important research field. Among these generative adversarial network models, a distinct advantage of CycleGAN is that it can transfer between multiple domains when the data is not paired. To approximate the effects of the texturing method with the characteristics of traditional Chinese painting—"Cun method", this paper proposes an image style transfer framework to realize the transfer from scenery photos to Chinese landscape paintings. We design a contour-enhancing translation branch, which effectively guides the transfer from photos to paintings with edge detection operators computing the gradient maps. Simulation results show that this method can convert real scenery photos to Chinese landscape paintings. The Inception Score shows that contour enhancement can make the generated set performs better on sensitivity to image edges. The Kernel Inception distance and Inception-based Structural Similarity between the generated image and the "Cun method" data set shows that contour enhancement can make the generated image closer to the "Cun method" effect. Compared with Kernel Inception distance and Frechet-Inception Distance, the Inception-based Structural Similarity proposed in this paper directly focuses on similarity, the similarities between the mean features of images generated by our model, and the "Cun method" set is 97.89%, and the composite similarity metric being 0.92. The method also performs better than the MUNIT, NiceGAN, CycleGAN, and U-GAT-IT reference models under the Neural Image Assessment metric. This indicates that the introduction of the edge operator makes the generated landscape paintings more aesthetic, especially in situations where scenery photos are rich in edge information.