Deep unsupervised pixelization
Deep unsupervised pixelization
复制标题
DOI：
10.1145/3272127.3275082
复制
发表时间：
2018-12
期刊：
影响因子：
--
通讯作者：
Chu Han;Q. Wen;Shengfeng He;Qianshu Zhu;Yinjie Tan;Guoqiang Han;T. Wong
中科院分区：
文献类型：
--
作者：
Chu Han;Q. Wen;Shengfeng He;Qianshu Zhu;Yinjie Tan;Guoqiang Han;T. Wong
在本文中，我们提出了一种新颖的像素化无监督学习方法。由于创建像素艺术的困难，为监督学习准备配对训练数据是不切实际的。相反，我们提出了一个无监督学习框架来规避这种困难。我们利用像素化和去像素化的双重性质，并在同一网络中以双向方式对这两个任务进行建模，并将输入本身作为训练监督。这两个任务被建模为一个级联网络，该网络由用于不同目的的三个阶段组成。 GridNet将输入图像转换为具有不同锯齿效果的多尺度网格结构图像。 PixelNet 与 GridNet 关联，合成具有锐利边缘和感知最佳局部结构的像素艺术。 DepixelNet连接之前的网络，旨在将像素化结果恢复为原始图像。为了无监督学习，提出了镜像损失来保持过程中特征表示的可逆性。此外，网络中还涉及对抗性损失、L1损失和梯度损失，以通过保留颜色正确性和平滑度来获得像素艺术。我们表明，与最先进的图像缩小方法相比，我们的技术可以合成更清晰、感知更合适的像素艺术。我们通过对许多图像进行大量实验来评估所提出的方法。所提出的方法在视觉质量和用户偏好方面优于最先进的方法。
In this paper, we present a novel unsupervised learning method for pixelization. Due to the difficulty in creating pixel art, preparing the paired training data for supervised learning is impractical. Instead, we propose an unsupervised learning framework to circumvent such difficulty. We leverage the dual nature of the pixelization and depixelization, and model these two tasks in the same network in a bi-directional manner with the input itself as training supervision. These two tasks are modeled as a cascaded network which consists of three stages for different purposes. GridNet transfers the input image into multi-scale grid-structured images with different aliasing effects. PixelNet associated with GridNet to synthesize pixel arts with sharp edges and perceptually optimal local structures. DepixelNet connects the previous network and aims to recover the pixelized result to the original image. For the sake of unsupervised learning, the mirror loss is proposed to hold the reversibility of feature representations in the process. In addition, adversarial, L1, and gradient losses are involved in the network to obtain pixel arts by retaining color correctness and smoothness. We show that our technique can synthesize crisper and perceptually more appropriate pixel arts than state-of-the-art image downscaling methods. We evaluate the proposed method with extensive experiments on many images. The proposed method outperforms state-of-the-art methods in terms of visual quality and user preference.