Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network
Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network
复制标题
卷积神经网络无参考质量评估立体图像的学习结构
DOI：
10.1016/j.patcog.2016.01.034
复制
发表时间：
2016-11-01
影响因子：
8
通讯作者：
Huang, Rui
中科院分区：
文献类型：
--
作者：
Zhang, Wei;Qu, Chenfei;Huang, Rui
在本文中，我们提出了学习立体图像的结构的卷积神经网络（CNN）的基础上，无参考质量评估。以立体图像中的图像块作为输入，所提出的CNN可以学习对人类感知敏感并代表感知质量评估的局部结构。通过将多个卷积层和最大池化层堆叠在一起，可以将较低卷积层中的学习结构组合并卷积到更高级别，以形成固定长度的表示。多层感知器（MLP）被进一步用于将学习的表示总结为最终值以指示立体图像块对的感知质量。对于不同的输入，设计了两种不同的CNN，即仅以来自差分图像的图像块作为输入的一列CNN，以及以来自左视图图像、右视图图像和差分图像的图像块作为输入的三列CNN。立体图像的CNN参数是基于大量的2D自然图像来学习和传输的。通过对公共LIVE phase-I、LIVE phase-II和IVC立体图像数据库的评估，该无参考度量达到了最先进的立体图像质量评估性能，甚至可以与现有的全参考质量度量相媲美。(C)2016爱思唯尔有限公司版权所有。
In this paper, we propose to learn the structures of stereoscopic image based on convolutional neural network (CNN) for no-reference quality assessment. Taking image patches from the stereoscopic images as inputs, the proposed CNN can learn the local structures which are sensitive to human perception and representative for perceptual quality evaluation. By stacking multiple convolution and max-pooling layers together, the learned structures in lower convolution layers can be composed and convolved to higher levels to form a fixed-length representation. Multilayer perceptron (MLP) is further employed to summarize the learned representation to a final value to indicate the perceptual quality of the stereo image patch pair. With different inputs, two different CNNs are designed, namely one-column CNN with only the image patch from the difference image as input, and three-column CNN with the image patches from left-view image, right-view image, and difference image as the input. The CNN parameters for stereoscopic images are learned and transferred based on the large number of 2D natural images. With the evaluation on public LIVE phase-I, LIVE phase-II, and IVC stereoscopic image databases, the proposed no-reference metric achieves the state-of-the-art performance for quality assessment of stereoscopic images, and is even competitive to existing full-reference quality metrics. (C) 2016 Elsevier Ltd. All rights reserved.