Pruning Early Exit Networks
Pruning Early Exit Networks
复制标题
修剪早期退出网络
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Erdem Koyuncu
中科院分区：
文献类型：
--
作者：
Alperen Görmez;Erdem Koyuncu
表现良好的深度学习模型通常有很高的计算成本。在本文中，我们结合了两种方法来降低计算成本，同时保持模型的高性能：修剪和早期退出网络。我们评估了两种修剪早期退出网络的方法：(1)一次修剪整个网络，(2)以有序的方式修剪基本网络和额外的线性分类器。实验结果表明，在一般情况下，一次性修剪整个网络是一种较好的策略。然而，在高准确率下，这两种方法具有相似的性能，这意味着修剪和早期退出过程可以在不损失最优性的情况下分离。
Deep learning models that perform well often have high computational costs. In this paper, we combine two approaches that try to reduce the computational cost while keeping the model performance high: pruning and early exit networks. We evaluate two approaches of pruning early exit networks: (1) pruning the entire network at once, (2) pruning the base network and additional linear classifiers in an ordered fashion. Experimental results show that pruning the entire network at once is a better strategy in general. However, at high accuracy rates, the two approaches have a similar performance, which implies that the processes of pruning and early exit can be separated without loss of optimality.