No-Reference Quality Assessment for 360-Degree Images by Analysis of Multifrequency Information and Local-Global Naturalness
No-Reference Quality Assessment for 360-Degree Images by Analysis of Multifrequency Information and Local-Global Naturalness
复制标题
DOI：
10.1109/tcsvt.2021.3081182
复制
发表时间：
2021-02
影响因子：
8.4
通讯作者：
Wei Zhou;Jiahua Xu;Qiuping Jiang;Zhibo Chen
中科院分区：
文献类型：
--
作者：
Wei Zhou;Jiahua Xu;Qiuping Jiang;Zhibo Chen
随着虚拟现实技术（VR）的应用越来越广泛，360度/全向图像（OIs）受到了人们的广泛关注。与传统的2D图像相比，OIs可以为消费者提供更身临其境的体验，受益于更高的分辨率和丰富的视野（fov）。此外，观察OIs通常是在头戴式显示器（HMD）中没有参考。因此，迫切需要针对360度图像设计一种高效的盲质量评价方法。本文基于人类视觉系统（HVS）的特点和VR视觉内容的观看过程，提出了一种基于多频信息和局部-全局自然度（MFILGN）的无参考全向图像质量评估（NR OIQA）算法。具体来说，利用视觉皮层的频率相关特性，我们首先利用离散Haar小波变换（DHWT）将投影的等矩形投影（ERP）映射分解成小波子带。然后，利用低频子带和高频子带的熵强度来测量信号的多频信息。除了考虑ERP地图的全局自然度外，由于浏览的fov，我们从每个视口图像中提取自然场景统计（NSS）特征作为局部自然度的度量。在提出的多频信息度量和局部-全局自然度度量的基础上，利用支持向量回归（SVR）作为最终的图像质量回归量，训练从视觉质量相关特征到人类评分的质量评价模型。据我们所知，该模型是第一个结合多频信息和图像自然度的360度图像无参考质量评估方法。在两个公开可用的OIQA数据库上的实验结果表明，我们提出的MFILGN优于最先进的全参考（FR）和NR方法。
360-degree/omnidirectional images (OIs) have received remarkable attention due to the increasing applications of virtual reality (VR). Compared to conventional 2D images, OIs can provide more immersive experiences to consumers, benefiting from the higher resolution and plentiful field of views (FoVs). Moreover, observing OIs is usually in a head-mounted display (HMD) without references. Therefore, an efficient blind quality assessment method, which is specifically designed for 360-degree images, is urgently desired. In this paper, motivated by the characteristics of the human visual system (HVS) and the viewing process of VR visual content, we propose a novel and effective no-reference omnidirectional image quality assessment (NR OIQA) algorithm by MultiFrequency Information and Local-Global Naturalness (MFILGN). Specifically, inspired by the frequency-dependent property of the visual cortex, we first decompose the projected equirectangular projection (ERP) maps into wavelet subbands by using discrete Haar wavelet transform (DHWT). Then, the entropy intensities of low-frequency and high-frequency subbands are exploited to measure the multifrequency information of OIs. In addition to considering the global naturalness of ERP maps, owing to the browsed FoVs, we extract the natural scene statistics (NSS) features from each viewport image as the measure of local naturalness. With the proposed multifrequency information measurement and local-global naturalness measurement, we utilize support vector regression (SVR) as the final image quality regressor to train the quality evaluation model from visual quality-related features to human ratings. To our knowledge, the proposed model is the first no-reference quality assessment method for 360-degree images that combines multifrequency information and image naturalness. Experimental results on two publicly available OIQA databases demonstrate that our proposed MFILGN outperforms state-of-the-art full-reference (FR) and NR approaches.