Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps-猫眼课题宝

DOI：

10.48550/arxiv.2209.02869

发表时间：

2022-09

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Alireza Ganjdanesh;Shangqian Gao;Heng Huang

卷积神经网络（CNN）压缩对于在资源有限的边缘设备中部署这些模型至关重要。现有的CNN信道修剪算法在复杂模型上取得了很大的成功。他们从不同的角度来处理修剪问题，并使用不同的指标来指导修剪过程。然而，这些衡量标准主要侧重于模型的“产出”或“权重”，而忽略了其“解释”信息。为了填补这一空白，我们建议从一个新的角度来解决通道修剪问题，利用模型的解释来引导修剪过程，从而利用模型的输入和输出的信息。然而，现有的解释方法不能得到部署，以实现我们的目标，因为它们要么是低效的修剪或可能预测不连贯的解释。我们通过引入一个选择器模型来解决这个挑战，该模型可以预测修剪模型的实时平滑显着性掩模。我们通过径向基函数（RBF）类函数将解释性掩模的分布参数化，以将自然图像的几何先验纳入我们的选择器模型的归纳偏差中。因此，我们可以获得紧凑的表示的解释，以减少我们的修剪方法的计算成本。我们利用我们的选择器模型，通过最大化修剪后的模型和原始模型的解释性表示的相似性来引导网络修剪。在CIFAR-10和ImageNet基准数据集上的大量实验证明了我们所提出的方法的有效性。我们的实现可以在\url{https：//github.com/Alii-Ganjj/InterpretationsSteeredPruning}上找到

Convolutional Neural Networks (CNNs) compression is crucial to deploying these models in edge devices with limited resources. Existing channel pruning algorithms for CNNs have achieved plenty of success on complex models. They approach the pruning problem from various perspectives and use different metrics to guide the pruning process. However, these metrics mainly focus on the model's `outputs' or `weights' and neglect its `interpretations' information. To fill in this gap, we propose to address the channel pruning problem from a novel perspective by leveraging the interpretations of a model to steer the pruning process, thereby utilizing information from both inputs and outputs of the model. However, existing interpretation methods cannot get deployed to achieve our goal as either they are inefficient for pruning or may predict non-coherent explanations. We tackle this challenge by introducing a selector model that predicts real-time smooth saliency masks for pruned models. We parameterize the distribution of explanatory masks by Radial Basis Function (RBF)-like functions to incorporate geometric prior of natural images in our selector model's inductive bias. Thus, we can obtain compact representations of explanations to reduce the computational costs of our pruning method. We leverage our selector model to steer the network pruning by maximizing the similarity of explanatory representations for the pruned and original models. Extensive experiments on CIFAR-10 and ImageNet benchmark datasets demonstrate the efficacy of our proposed method. Our implementations are available at \url{https://github.com/Alii-Ganjj/InterpretationsSteeredPruning}

Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps

文献摘要

相似文献