CAREER: Content-Based Image and Video Coding Using Higher-Level Models of Human Vision

当前的图像和视频编码方法之所以有效，很大程度上是因为它们利用了人类视觉系统（HVS）的低级方面。最主要的策略是将错误放置在可以更好地隐藏压缩伪影的区域，这种方法可以通过早期/低级HVS处理的计算模型来指导。这种方法的固有假设是，消费者正在寻找图像存在的失真。然而，在现实中，消费者看到的是可能存在失真的图像，这是一个完全不同的感知任务，需要一个完全不同的HVS模型。下一代编码方案，可以考虑更高层次的方面，如内容自适应掩蔽；跨越空间、频率和时间的感知重要性；认识的要素；具有显著减少存储和带宽需求的潜力，同时最大化视觉质量和整体多媒体体验。在这个项目中，研究者研究压缩伪影如何影响HVS处理和解释图像和视频的能力。主要研究了三个方面：(1)考虑图像识别的视觉掩蔽新模型；(2)数据量化的外观保持策略；(3)基于质量评定实验与眼动追踪相结合的视觉认知规律的分析与量化策略。这项研究与教育组成部分相结合，促进学生将人类视觉知识应用于工程问题的发展。两个新的跨学科研究生水平课程，一个跨学科的夏季研讨会，以及本科生研究项目和课程改革提供给学生。两个多媒体驱动的比赛，让K-12学生和本科生接触到图像处理研究。

Abstract #1054687Current methods of image and video coding are effective largely because they capitalize on low-level aspects of the human visual system (HVS). The single most predominant strategy is to place the errors into regions which can better hide the compression artifacts, an approach which can be guided by computational models of early/low-level HVS processing. The inherent assumption in this approach is that the consumer is looking for the distortion in the presence of the image. However, in actuality, the consumer is looking at the image in the presence of possible distortion, which is a fundamentally different perceptual task that requires a fundamentally different HVS model. Next-generation coding schemes which can take into account higher-level aspects such as content-adaptive masking; perceptual importance across space, frequency, and time; and elements of cognition; have the potential to dramatically reduce storage and bandwidth requirements while maximizing visual quality and the overall multimedia experience. In this project, the investigator researches how compression artifacts influence the HVS's ability to process and interpret images and video. Three main areas are investigated: (1) new models of visual masking which take into account image recognition; (2) appearance-preserving strategies of data quantization; and (3) analysis and quantization strategies which honor rules of visual cognition derived from quality-rating experiments coupled with eye-tracking. This research is integrated with an educational component that promotes student development in applying knowledge of human vision to engineering problems. Two new interdisciplinary graduate-level courses, an interdisciplinary summer workshop, and undergraduate research projects and curriculum reform are made available to students. Two multimedia-driven competitions that expose K-12 students and undergraduates to image-processing research are also made available.