Approximated Oracle Filter Pruning for Destructive CNN Width Optimization
Approximated Oracle Filter Pruning for Destructive CNN Width Optimization
复制标题
DOI：
--
复制
发表时间：
2019-05
期刊：
影响因子：
--
通讯作者：
Xiaohan Ding;Guiguang Ding;Yuchen Guo;J. Han;C. Yan
中科院分区：
文献类型：
--
作者：
Xiaohan Ding;Guiguang Ding;Yuchen Guo;J. Han;C. Yan
设计和运行卷积神经网络（CNN）并不容易，这是由于：1）找到滤波器的最佳数量（即，在给定架构的情况下，每层的宽度）是棘手的;以及2）CNN的计算强度阻碍了在计算受限的设备上的部署。Oracle Pruning旨在从经过良好训练的CNN中删除不重要的过滤器，该过滤器通过依次消融过滤器并评估模型来估计过滤器的重要性，从而提供高精度，但具有无法忍受的时间复杂度，并且需要给定的结果宽度但不能自动找到它。为了解决这些问题，我们提出了近似Oracle过滤器修剪（AOFP），其以二分搜索方式保持搜索最不重要的过滤器，通过随机屏蔽过滤器来进行修剪尝试，累积所产生的错误，并经由多路径框架来微调模型。由于AOFP可以在多个层上同时修剪，因此我们可以以可接受的时间成本，可忽略的准确性下降，并且没有启发式知识来修剪现有的非常深的CNN，或者重新设计一个具有更高准确性和更快推理的模型
It is not easy to design and run Convolutional Neural Networks (CNNs) due to: 1) finding the optimal number of filters (i.e., the width) at each layer is tricky, given an architecture; and 2) the computational intensity of CNNs impedes the deployment on computationally limited devices. Oracle Pruning is designed to remove the unimportant filters from a well-trained CNN, which estimates the filters’ importance by ablating them in turn and evaluating the model, thus delivers high accuracy but suffers from intolerable time complexity, and requires a given resulting width but cannot automatically find it. To address these problems, we propose Approximated Oracle Filter Pruning (AOFP), which keeps searching for the least important filters in a binary search manner, makes pruning attempts by masking out filters randomly, accumulates the resulting errors, and finetunes the model via a multi-path framework. As AOFP enables simultaneous pruning on multiple layers, we can prune an existing very deep CNN with acceptable time cost, negligible accuracy drop, and no heuristic knowledge, or re-design a model which exerts higher accuracy and faster inference