DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation
DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation
复制标题
DOI：
10.1007/978-3-030-58580-8_35
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Xuefei Ning;Tianchen Zhao;Wenshuo Li;Peng Lei;Yu Wang;Huazhong Yang
中科院分区：
文献类型：
--
作者：
Xuefei Ning;Tianchen Zhao;Wenshuo Li;Peng Lei;Yu Wang;Huazhong Yang
Budgeted pruning is the problem of pruning under resource constraints. In budgeted pruning, how to distribute the resources across layers (i.e., sparsity allocation) is the key problem. Traditional methods solve it by discretely searching for the layer-wise pruning ratios, which lacks efficiency. In this paper, we propose Differentiable Sparsity Allocation (DSA), an efficient end-to-end budgeted pruning flow. Utilizing a noveldifferentiable pruning process, DSA finds the layer-wise pruning ratios withgradient-based optimization. It allocates sparsity in continuous space, which is more efficient than methods based on discrete evaluation and search. Furthermore, DSA could work in apruning-from-scratchmanner, whereas traditional budgeted pruning methods are applied to pre-trained models. Experimental results on CIFAR-10 and ImageNet show that DSA could achieve superior performance than current iterative budgeted pruning methods, and shorten the time cost of the overall pruning process by at least 1.5in the meantime.
