Interactive Deep Colorization and its Application for Image Compression
Interactive Deep Colorization and its Application for Image Compression
复制标题
交互式深色着色及其在图像压缩中的应用
DOI：
10.1109/tvcg.2020.3021510
复制
发表时间：
2020-09
影响因子：
5.2
通讯作者：
Ladislav Kavan
中科院分区：
文献类型：
--
作者：
Yi Xiao;Jin Wu;Jie Zhang;Peiyao Zhou;Yan Zheng;Chi-Sing Leung;Ladislav Kavan
最近基于深度学习的方法在将灰度图像转换为彩色图像方面显示出了前景。然而，它们中的大多数仅允许有限的用户输入（无输入、仅全局输入或仅本地输入）来控制输出彩色图像。可能的困难在于如何区分不同输入的影响。为了解决这个问题，我们提出了一种两阶段深度着色方法，允许用户通过灵活设置全局输入和局部输入来控制结果。关键步骤包括通过提取 <inline-formula><tex-math notation="LaTeX">$K$</tex-math><alternatives><mml:math><mml:mi>K</mml:mi></mml:math><inline-graphic 来启用颜色主题作为全局输入 xlink:href="xiao-ieq1-3021510.gif"/></alternatives></inline-formula> 表示颜色并生成 <inline-formula><tex-math notation="LaTeX">$K$</tex-math><alternatives><mml:math><mml:mi>K</mml:mi></mml:math><inline-graphic xlink:href="xiao-ieq2-3021510.gif"/></alternatives></inline-formula> - 用于定义全局主题损失的颜色映射，并设计一个损失函数来区分不同输入的影响，而不会造成伪影。我们还提出了一种颜色主题推荐方法来帮助用户选择颜色主题。基于着色模型，我们进一步提出了一种图像压缩方案，该方案支持单个网络中的可变压缩比。着色实验表明，我们的方法只需少量输入即可灵活控制着色结果，并生成最先进的结果。压缩实验表明，与最先进的方法相比，我们的方法在相同的压缩比下实现了更高的图像质量。
Recent methods based on deep learning have shown promise in converting grayscale images to colored ones. However, most of them only allow limited user inputs (no inputs, only global inputs, or only local inputs), to control the output colorful images. The possible difficulty lies in how to differentiate the influences of different inputs. To solve this problem, we propose a two-stage deep colorization method allowing users to control the results by flexibly setting global inputs and local inputs. The key steps include enabling color themes as global inputs by extracting <inline-formula><tex-math notation="LaTeX">$K$</tex-math><alternatives><mml:math><mml:mi>K</mml:mi></mml:math><inline-graphic xlink:href="xiao-ieq1-3021510.gif"/></alternatives></inline-formula> mean colors and generating <inline-formula><tex-math notation="LaTeX">$K$</tex-math><alternatives><mml:math><mml:mi>K</mml:mi></mml:math><inline-graphic xlink:href="xiao-ieq2-3021510.gif"/></alternatives></inline-formula>-color maps to define a global theme loss, and designing a loss function to differentiate the influences of different inputs without causing artifacts. We also propose a color theme recommendation method to help users choose color themes. Based on the colorization model, we further propose an image compression scheme, which supports variable compression ratios in a single network. Experiments on colorization show that our method can flexibly control the colorized results with only a few inputs and generate state-of-the-art results. Experiments on compression show that our method achieves much higher image quality at the same compression ratio when compared to the state-of-the-art methods.