Adaptive Instance Distillation for Object Detection in Autonomous Driving
Adaptive Instance Distillation for Object Detection in Autonomous Driving
复制标题
DOI：
10.1109/icpr56361.2022.9956165
复制
发表时间：
2022-01
期刊：
影响因子：
--
通讯作者：
Qizhen Lan;Qing Tian
中科院分区：
文献类型：
--
作者：
Qizhen Lan;Qing Tian
近年来，知识蒸馏（KD）已被广泛用于获得有效的模型。通过模仿大型教师模型，轻量级学生模型可以以更高的效率实现相当的性能。然而，大多数现有的知识提取方法都集中在分类任务。只有有限数量的研究将知识蒸馏应用于物体检测，特别是在时间敏感的自动驾驶场景中。在本文中，我们提出了自适应实例蒸馏（AID）有选择地传授教师的知识，以提高知识蒸馏的性能。与以往的KD方法，平等地对待所有的实例，我们的AID可以用心调整的基础上，教师模型的预测损失的实例的蒸馏权重。通过在KITTI和COCO流量数据集上的实验，验证了该方法的有效性。结果表明，我们的方法提高了最先进的注意力引导和非局部蒸馏方法的性能，并在单级和两级检测器上实现了更好的蒸馏结果。与基线相比，我们的AID导致单级和两级探测器的平均mAP分别增加了2.7%和2.1%。此外，我们的AID也被证明是有用的自蒸馏，以提高教师模型的性能。
In recent years, knowledge distillation (KD) has been widely used to derive efficient models. Through imitating a large teacher model, a lightweight student model can achieve comparable performance with more efficiency. However, most existing knowledge distillation methods are focused on classification tasks. Only a limited number of studies have applied knowledge distillation to object detection, especially in time-sensitive autonomous driving scenarios. In this paper, we propose Adaptive Instance Distillation (AID) to selectively impart teacher’s knowledge to the student to improve the performance of knowledge distillation. Unlike previous KD methods that treat all instances equally, our AID can attentively adjust the distillation weights of instances based on the teacher model’s prediction loss. We verified the effectiveness of our AID method through experiments on the KITTI and the COCO traffic datasets. The results show that our method improves the performance of state-of-the-art attention-guided and non-local distillation methods and achieves better distillation results on both single-stage and two-stage detectors. Compared to the baseline, our AID led to an average of 2.7% and 2.1% mAP increases for single-stage and two-stage detectors, respectively. Furthermore, our AID is also shown to be useful for self-distillation to improve the teacher model’s performance.