CRII: RI: Learning with Low-Quality Visual Data: Handling Both Passive and Active Degradations

该项目的重点是有效和稳健地利用低质量（LQ）视觉数据进行计算机视觉任务。虽然目前大多数计算机视觉系统都是为高质量的视觉数据而设计的，这些数据是从“清晰”的环境中收集的，在这种环境中，对象可以很好地观察到，而不会发生显著的衰减或改变，但可靠的视觉系统必须考虑到来自不受约束的环境的整个退化范围。由于视觉数据采集和处理流水线带来的各种退化，无处不在的LQ视觉数据在实际应用中会显著恶化模型性能。该项目的成果可以广泛地惠及各种现实应用，如视频监控、自动/辅助驾驶、机器人和医学图像分析，其中LQ视觉数据构成了主要的性能和可靠性瓶颈。本研究将常见的降级分为两种类型：由不可控的环境因素（如恶劣天气和低光照）引起的“被动降级”;以及以可控方式故意引入以满足某些预算要求（如有损压缩）的“主动降级”。该项目将主要解决两个重要的技术问题：i）如何克服被动降级，并使用端到端深度学习模型在LQ视频数据上实现更强大的高级任务性能;以及ii）如何正确引入和控制主动降级，以生成所需形式的LQ数据，既满足某些预算要求，又保持目标任务效用，使用深度对抗学习模型。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project is focused on effectively and robustly exploiting low-quality (LQ) visual data for computer vision tasks. While most current computer vision systems are designed for high-quality visual data, collected from "clear" environments where subjects are well observable without significant attenuation or alteration, a dependable vision system must reckon with the entire spectrum of degradations from unconstrained environments. With various degradations arising from the visual data acquisition and processing pipeline, the ubiquitous LQ visual data can dramatically deteriorate the model performance in practice. The project outcome can broadly benefit a variety of real-world applications, such as video surveillance, autonomous/assisted driving, robotics and medical image analysis, where LQ visual data has constituted major performance and reliability bottlenecks. This research categorizes common degradations into the two types: "passive degradations" that are caused by uncontrollable environment factors (such as bad weather and low light); and "active degradations" that are intentionally introduced in a controllable way to meet certain budget requirements (such as lossy compression). The project will mainly addresses two important technical questions: i) how to overcome passive degradations and achieve more robust high-level task performance on LQ video data, using end-to-end deep learning models; and ii) how to properly introduce and control active degradations to generate the desired form of LQ data, that both satisfies certain budget requirements and maintains the target task utility, using deep adversarial learning models. The resulting new techniques are to be verified on application examples such as video recognition, video annotation, video compression, and de-identified video data sharing for recognition purpose.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.