Semantic-aware automatic image colorization via unpaired cycle-consistent self-supervised network
Semantic-aware automatic image colorization via unpaired cycle-consistent self-supervised network
复制标题
通过不成对的循环实现语义感知的自动图像着色——一致的自监督网络
DOI：
10.1002/int.22667
复制
发表时间：
2021-09-19
影响因子：
7
通讯作者：
Wang, Mingwen
中科院分区：
文献类型：
--
作者：
Xiao, Yuxuan;Jiang, Aiwen;Wang, Mingwen
没有人工干预的自动图像着色是一个病态的和固有的模糊问题。大多数现有的方法都专注于将彩色化公式化为回归问题，并通过深度神经网络学习从灰度到颜色的参数映射。由于颜色-灰度空间的多模态性，在许多应用中，不需要恢复精确的地面实况颜色。基于成对像素到像素学习的算法缺乏合理性。然后提出诸如颜色空间转换技术的技术来避免这种直接像素学习。然而，颜色空间转换后的着色结果是生硬和不自然的。在本文中，我们认为一个合理的解决方案是产生一些彩色的结果，看起来自然。无论要为区域分配什么颜色，着色的区域都应该在语义和空间上一致。本文提出了一种基于非成对循环一致自监督网络的语义感知的自动彩色化模型。引入低层单色损失、感知同一性损失和高层语义一致性损失以及对抗性损失来指导网络自训练。我们在PASCAL VOC 2012中随机选择的子集上训练和测试我们的模型。实验结果表明，与现有的方法相比，该模型可以获得更有说服力和更优越的上级结果。相关源代码可在。
Automatic image colorization without manual interventions is an ill-conditioned and inherently ambiguous problem. Most of existing methods focus on formulating colorization as a regression problem and learn parametric mappings from grayscale to color through deep neural networks. Due to the multimodalities of color-grayscale space, in many applications, it is not required to recover exact ground-truth color. Pair-wise pixel-to-pixel learning-based algorithms lack rationality. Techniques such as color space conversion techniques are then proposed to avoid such direct pixel learning. However, the coloring results after color space conversion are blunt and unnatural. In this paper, we hold viewpoints that a reasonable solution is to generate some colorized result that looks natural. No matter what color a region is to be assigned, the colorized region should be semantically and spatially consistent. In this paper, we propose an effective semantic-aware automatic colorization model via unpaired cycle-consistent self-supervised network. Low-level monochrome loss, perceptual identity loss and high-level semantic-consistence loss, together with adversarial loss, are introduced to guide network self-training. We train and test our model on randomly selected subsets from PASCAL VOC 2012. The experimental results including human subjective studies demonstrate that, compared with state-of-the-art methods, our proposed model can achieve more convincing and superior results. Relevant source code is available at .