One-shot Network Pruning at Initialization with Discriminative Image Patches
One-shot Network Pruning at Initialization with Discriminative Image Patches
复制标题
DOI：
10.48550/arxiv.2209.05683
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Yinan Yang;Yi Ji;Yu Wang;Heng Qi;Jien Kato
中科院分区：
文献类型：
--
作者：
Yinan Yang;Yi Ji;Yu Wang;Heng Qi;Jien Kato
初始化时一次性网络剪枝(OPAI)是降低网络剪枝成本的有效方法。最近，越来越多的人认为，数据在OPAI中是不必要的。然而，在两种具有代表性的OPAI方法--SNIP和GRAPH中的消融实验中，我们得到了相反的结论。具体地说，我们发现信息量大的数据对于提高修剪性能至关重要。在本文中，我们提出了两种新的方法：区分单镜头网络剪枝(DOP)和超级缝合，利用高层视觉区分图像块对网络进行剪枝。我们的贡献如下。(1)大量实验表明，OPAI依赖于数据。(2)在基准ImageNet上，超级拼接的性能明显优于原始的OPAI方法，尤其是在高度压缩的模型上。
One-shot Network Pruning at Initialization (OPaI) is an effective method to decrease network pruning costs. Recently, there is a growing belief that data is unnecessary in OPaI. However, we obtain an opposite conclusion by ablation experiments in two representative OPaI methods, SNIP and GraSP. Specifically, we find that informative data is crucial to enhancing pruning performance. In this paper, we propose two novel methods, Discriminative One-shot Network Pruning (DOP) and Super Stitching, to prune the network by high-level visual discriminative image patches. Our contributions are as follows. (1) Extensive experiments reveal that OPaI is data-dependent. (2) Super Stitching performs significantly better than the original OPaI method on benchmark ImageNet, especially in a highly compressed model.