Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification
Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification
复制标题
DOI：
10.1145/2897824.2925974
复制
发表时间：
2016-07-01
影响因子：
6.2
通讯作者：
Ishikawa, Hiroshi
中科院分区：
文献类型：
--
作者：
Iizuka, Satoshi;Simo-Serra, Edgar;Ishikawa, Hiroshi
我们提出了一种新的技术来自动着色灰度图像，结合全局先验和局部图像特征。基于卷积神经网络，我们的深度网络具有融合层，使我们能够优雅地将依赖于小图像块的局部信息与使用整个图像计算的全局先验信息合并。整个框架，包括全局和局部先验以及着色模型，都是以端到端的方式进行训练的。此外，我们的架构可以处理任何分辨率的图像，这与大多数基于CNN的现有方法不同。我们利用现有的大规模场景分类数据库来训练我们的模型，利用数据集的类标签来更有效地和有区别地学习全局先验。我们通过用户研究验证了我们的方法，并与最先进的方法进行了比较，我们显示出显着的改进。此外，我们在许多不同类型的图像上广泛地演示了我们的方法，包括一百多年前的黑白白色摄影，并显示了逼真的着色。
We present a novel technique to automatically colorize grayscale images that combines both global priors and local image features. Based on Convolutional Neural Networks, our deep network features a fusion layer that allows us to elegantly merge local information dependent on small image patches with global priors computed using the entire image. The entire framework, including the global and local priors as well as the colorization model, is trained in an end-to-end fashion. Furthermore, our architecture can process images of any resolution, unlike most existing approaches based on CNN. We leverage an existing large-scale scene classification database to train our model, exploiting the class labels of the dataset to more efficiently and discriminatively learn the global priors. We validate our approach with a user study and compare against the state of the art, where we show significant improvements. Furthermore, we demonstrate our method extensively on many different types of images, including black-and-white photography from over a hundred years ago, and show realistic colorizations.