CRII: RI: Learning with Low-Quality Visual Data: Handling Both Passive and Active Degradations

本项目的重点是有效和稳健地利用低质量（LQ）视觉数据进行计算机视觉任务。虽然目前大多数计算机视觉系统都是为高质量的视觉数据而设计的，这些数据是从“清晰”的环境中收集的，在这种环境中，受试者可以很好地观察到，没有明显的衰减或变化，但一个可靠的视觉系统必须考虑到来自无约束环境的整个退化范围。由于视觉数据的采集和处理管道会产生各种各样的退化，在实际应用中，无处不在的LQ视觉数据会严重降低模型的性能。该项目的成果可以广泛受益于各种实际应用，如视频监控、自动/辅助驾驶、机器人和医疗图像分析，在这些应用中，LQ视觉数据构成了主要的性能和可靠性瓶颈。这项研究将常见的退化分为两种类型：由不可控的环境因素（如恶劣天气和光线不足）引起的“被动退化”；以及“主动降级”，这是以可控的方式引入的，以满足某些预算要求（例如有损压缩）。该项目将主要解决两个重要的技术问题：i)如何使用端到端深度学习模型克服被动退化并在LQ视频数据上实现更强大的高级任务性能；ii)如何适当地引入和控制主动退化，以生成期望的LQ数据形式，既满足一定的预算要求，又保持目标任务效用，使用深度对抗学习模型。所得到的新技术将在视频识别、视频注释、视频压缩和以识别为目的的去识别视频数据共享等应用实例中得到验证。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project is focused on effectively and robustly exploiting low-quality (LQ) visual data for computer vision tasks. While most current computer vision systems are designed for high-quality visual data, collected from "clear" environments where subjects are well observable without significant attenuation or alteration, a dependable vision system must reckon with the entire spectrum of degradations from unconstrained environments. With various degradations arising from the visual data acquisition and processing pipeline, the ubiquitous LQ visual data can dramatically deteriorate the model performance in practice. The project outcome can broadly benefit a variety of real-world applications, such as video surveillance, autonomous/assisted driving, robotics and medical image analysis, where LQ visual data has constituted major performance and reliability bottlenecks. This research categorizes common degradations into the two types: "passive degradations" that are caused by uncontrollable environment factors (such as bad weather and low light); and "active degradations" that are intentionally introduced in a controllable way to meet certain budget requirements (such as lossy compression). The project will mainly addresses two important technical questions: i) how to overcome passive degradations and achieve more robust high-level task performance on LQ video data, using end-to-end deep learning models; and ii) how to properly introduce and control active degradations to generate the desired form of LQ data, that both satisfies certain budget requirements and maintains the target task utility, using deep adversarial learning models. The resulting new techniques are to be verified on application examples such as video recognition, video annotation, video compression, and de-identified video data sharing for recognition purpose.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.