Gradient-Guided Knowledge Distillation for Object Detectors
Gradient-Guided Knowledge Distillation for Object Detectors
复制标题
DOI：
10.1109/wacv57701.2024.00049
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
Qizhen Lan;Qingze Tian
中科院分区：
文献类型：
--
作者：
Qizhen Lan;Qingze Tian
深度学习模型在对象检测中表现出了显着的成功，但是它们的复杂性和计算强度构成了将它们部署在现实世界应用中的障碍（例如，自动驾驶感知）。但是，只有少数KD方法处理对象检测最终检测。在本文中，我们提出了一种新的知识蒸馏的方法为了从老师那里学习最相关的功能。 Kitti和可可交通数据集的实验证明了我们的方法在对象检测方面的知识蒸馏效率。各种最新的KD方法。
Deep learning models have demonstrated remarkable success in object detection, yet their complexity and computational intensity pose a barrier to deploying them in real-world applications (e.g., self-driving perception). Knowledge Distillation (KD) is an effective way to derive efficient models. However, only a small number of KD methods tackle object detection. Also, most of them focus on mimicking the plain features of the teacher model but rarely consider how the features contribute to the final detection. In this paper, we propose a novel approach for knowledge distillation in object detection, named Gradient-guided Knowledge Distillation (GKD). Our GKD uses gradient information to identify and assign more weights to features that significantly impact the detection loss, allowing the student to learn the most relevant features from the teacher. Furthermore, we present bounding-box-aware multi-grained feature imitation (BMFI) to further improve the KD performance. Experiments on the KITTI and COCO-Traffic datasets demonstrate our method’s efficacy in knowledge distillation for object detection. On one-stage and two-stage detectors, our GKD-BMFI leads to an average of 5.1% and 3.8% mAP improvement, respectively, beating various state-of-the-art KD methods. Our codes are available at: https://github.com/lanqz7766/GKD.