New Measurements Reveal Weaknesses of Image Quality Metrics in Evaluating Graphics Artifacts
New Measurements Reveal Weaknesses of Image Quality Metrics in Evaluating Graphics Artifacts
复制标题
DOI：
10.1145/2366145.2366166
复制
发表时间：
2012-11-01
影响因子：
6.2
通讯作者：
Seidel, Hans-Peter
中科院分区：
文献类型：
--
作者：
Cadik, Martin;Herzog, Robert;Seidel, Hans-Peter
对于许多图形应用程序来说，可靠地检测全局照明和以局部失真地图的形式呈现伪影是很重要的。虽然已经为这项任务开发了许多质量度量，但它们通常是针对压缩/传输伪像进行调整的，并且没有在合成CG图像的背景下进行评估。在这项工作中，我们运行了两个实验，其中观察者使用画笔绘制界面来分别在存在/不存在高质量参考图像的情况下直接标记具有明显/令人反感的失真的图像区域。收集的数据显示，有参照和无参照的观察者标记之间具有较高的相关性。此外，我们要求苛刻的每像素图像质量数据集显示了简单(PSNR、MSE、SCIE-Lab)和高级(SSIM、MS-SSIM、HDR-VDP-2)质量指标的弱点。最大的问题是对亮度和对比度变化的过度敏感，对近能见度阈值失真的校准，缺乏对合理/不合理照明的区分，以及多尺度度量失真的空间定位不佳。我们相信，我们的数据集在改进现有质量度量方面具有进一步的潜力，但也在分析渲染失真的显著程度以及在给定有无参考数据的情况下调查视觉等价性方面具有进一步的潜力。
Reliable detection of global illumination and rendering artifacts in the form of localized distortion maps is important for many graphics applications. Although many quality metrics have been developed for this task, they are often tuned for compression/transmission artifacts and have not been evaluated in the context of synthetic CG-images. In this work, we run two experiments where observers use a brush-painting interface to directly mark image regions with noticeable/objectionable distortions in the presence/absence of a high-quality reference image, respectively. The collected data shows a relatively high correlation between the with-reference and no-reference observer markings. Also, our demanding perpixel image-quality datasets reveal weaknesses of both simple (PSNR, MSE, sCIE-Lab) and advanced (SSIM, MS-SSIM, HDR-VDP-2) quality metrics. The most problematic are excessive sensitivity to brightness and contrast changes, the calibration for near visibility-threshold distortions, lack of discrimination between plausible/implausible illumination, and poor spatial localization of distortions for multi-scale metrics. We believe that our datasets have further potential in improving existing quality metrics, but also in analyzing the saliency of rendering distortions, and investigating visual equivalence given our with-and no-reference data.