R-YOLO: A Robust Object Detector in Adverse Weather
R-YOLO: A Robust Object Detector in Adverse Weather
复制标题
DOI：
10.1109/tim.2022.3229717
复制
发表时间：
2023
影响因子：
5.6
通讯作者：
Lucai Wang;Hongda Qin;Xuanyu Zhou;Xiao Lu;Fengting Zhang
中科院分区：
文献类型：
--
作者：
Lucai Wang;Hongda Qin;Xuanyu Zhou;Xiao Lu;Fengting Zhang
在恶劣天气下实时高效地学习鲁棒的物体检测器对于自动驾驶系统的视觉感知任务非常重要。在本文中，我们提出了一个框架，将 YOLO 改进为鲁棒检测器，表示为 R(obust)-YOLO，无需在恶劣天气下进行注释。考虑到正常天气图像和恶劣天气图像之间的分布差距，我们的框架由图像准翻译网络（QTNet）和特征校准网络（FCNet）组成，用于逐渐使正常天气域适应恶劣天气域。具体来说，我们使用简单而有效的 QTNet 来生成继承正常天气域中的注释并插入两个域之间的间隙的图像。然后，在 FCNet 中，我们提出了两种基于对抗学习的特征校准模块，以局部到全局的方式有效地对齐两个域中的特征表示。有了这样的学习框架，我们的R-YOLO并没有改变原来的YOLO结构，因此它适用于所有YOLO系列检测器。我们的 R-YOLOv3、R-YOLOv5 和 R-YOLOX 在雾天和雨天数据集上的大量实验结果表明，我们的方法优于其他以去雾/去雨为预处理步骤的检测器和其他基于无监督域适应（UDA）的检测器，这证实了我们的方法仅利用未标记的恶劣天气图像来提高鲁棒性的有效性。我们的代码和预训练模型可在以下网址获取：https://github.com/qinhongda8/R-YOLO。
Learning a robust object detector in adverse weather with real-time efficiency is of great importance for the visual perception task for autonomous driving systems. In this article, we propose a framework to improve the YOLO to a robust detector, denoted as R(obust)-YOLO, without the need for annotations in adverse weather. Considering the distribution gap between the normal weather images and the adverse weather images, our framework consists of an image quasi-translation network (QTNet) and a feature calibration network (FCNet) for adapting the normal weather domain to the adverse weather domain gradually. Specifically, we use the simple yet effective QTNet for generating images that inherit the annotations in the normal weather domain and interpolate the gap between the two domains. Then, in FCNet, we propose two kinds of adversarial-learning-based feature calibration modules to effectively align the feature representations in two domains in a local-to-global manner. With such a learning framework, our R-YOLO does not change the original YOLO structure, and thus it is applicable to all the YOLO-series detectors. Extensive experimental results of our R-YOLOv3, R-YOLOv5, and R-YOLOX on both the hazy and rainy datasets show that our method outperforms other detectors with dehaze/derain as the preprocessing step and other unsupervised domain adaptation (UDA)-based detectors, which confirms the effectiveness of our method on improving the robustness by only leveraging the unlabeled adverse weather images. Our code and pretrained models are available at: https://github.com/qinhongda8/R-YOLO.