Enhance Visual Recognition Under Adverse Conditions via Deep Networks
Enhance Visual Recognition Under Adverse Conditions via Deep Networks
复制标题
利用深度网络增强恶劣环境下的视觉识别
DOI：
10.1109/tip.2019.2908802
复制
发表时间：
2019-09-01
影响因子：
10.6
通讯作者：
Huang, Thomas S.
中科院分区：
文献类型：
--
作者：
Liu, Ding;Cheng, Bowen;Huang, Thomas S.
由于图像采集、传输或存储过程中普遍存在质量失真，因此恶劣条件下的视觉识别是一个非常重要且具有挑战性的具有很高实用价值的问题。虽然深度神经网络已分别在低质量图像恢复和高质量图像识别任务中得到广泛应用，但对极低质量图像识别这一重要问题的研究却很少。本文提出了一种基于深度学习的框架，用于使用鲁棒的不利预训练或其积极变体来提高不利条件下图像和视频识别模型的性能。鲁棒的逆向预训练算法利用了预训练的力量，并推广了传统的无监督预训练和数据增强方法。我们进一步开发了一种迁移学习方法来应对未知不利条件的现实数据集。所提出的框架在多个图像和视频识别基准上进行了综合评估，并在各种单一或混合不利条件下获得了显着的性能改进。我们的可视化和分析进一步增加了结果的可解释性。
Visual recognition under adverse conditions is a very important and challenging problem of high practical value, due to the ubiquitous existence of quality distortions during image acquisition, transmission, or storage. While deep neural networks have been extensively exploited in the techniques of low-quality image restoration and high-quality image recognition tasks, respectively, few studies have been done on the important problem of recognition from very low-quality images. This paper proposes a deep learning-based framework for improving the performance of image and video recognition models under adverse conditions, using robust adverse pre-training or its aggressive variant. The robust adverse pre-training algorithms leverage the power of pre-training and generalize the conventional unsupervised pre-training and data augmentation methods. We further develop a transfer learning approach to cope with real-world datasets of unknown adverse conditions. The proposed framework is comprehensively evaluated on a number of image and video recognition benchmarks, and obtains significant performance improvements under various single or mixed adverse conditions. Our visualization and analysis further add to the explainability of the results.