Attention-Aware Multi-Stroke Style Transfer
Attention-Aware Multi-Stroke Style Transfer
复制标题
DOI：
10.1109/cvpr.2019.00156
复制
发表时间：
2019-01
期刊：
影响因子：
--
通讯作者：
Y. Yao;Jianqiang Ren;Xuansong Xie;Weidong Liu;Yong-Jin Liu;Jun Wang
中科院分区：
文献类型：
--
作者：
Y. Yao;Jianqiang Ren;Xuansong Xie;Weidong Liu;Yong-Jin Liu;Jun Wang
神经风格迁移已经引起了学术界和工业界的广泛关注。虽然视觉效果和效率都有了明显的提高，但现有的方法无法协调内容图像和风格化图像之间视觉注意力的空间分布，也无法通过不同的笔触呈现不同层次的细节。在本文中，我们通过开发一个注意意识的多笔画风格迁移模型来解决这些限制。我们首先提出将自注意机制组装成一个风格不可知的重建自编码器框架，从中可以导出内容图像的注意图。通过对内容特征和样式特征进行多尺度的样式交换，生成反映不同笔画模式的多个特征映射。进一步提出了一种灵活的融合策略，将注意图中的显著特征融合到输出图像的不同空间区域中，使多个笔画模式和谐地融合在一起。我们展示了我们的方法的有效性，以及与最先进的方法生成具有多种笔画模式的可比风格化图像。
Neural style transfer has drawn considerable attention from both academic and industrial field. Although visual effect and efficiency have been significantly improved, existing methods are unable to coordinate spatial distribution of visual attention between the content image and stylized image, or render diverse level of detail via different brush strokes. In this paper, we tackle these limitations by developing an attention-aware multi-stroke style transfer model. We first propose to assemble self-attention mechanism into a style-agnostic reconstruction autoencoder framework, from which the attention map of a content image can be derived. By performing multi-scale style swap on content features and style features, we produce multiple feature maps reflecting different stroke patterns. A flexible fusion strategy is further presented to incorporate the salient characteristics from the attention map, which allows integrating multiple stroke patterns into different spatial regions of the output image harmoniously. We demonstrate the effectiveness of our method, as well as generate comparable stylized images with multiple stroke patterns against the state-of-the-art methods.