Growing Efficient Deep Networks by Structured Continuous Sparsification
Growing Efficient Deep Networks by Structured Continuous Sparsification
复制标题
DOI：
--
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Xin Yuan;Pedro H. P. Savarese;M. Maire
中科院分区：
文献类型：
--
作者：
Xin Yuan;Pedro H. P. Savarese;M. Maire
我们开发了一种训练深度网络的方法，同时动态调整其架构，由准确性和稀疏性目标的原则性组合驱动。与传统的修剪方法不同，我们的方法采用离散网络结构优化的逐渐连续松弛，然后对稀疏子网络进行采样，从而能够以生长和修剪的方式训练高效的深度网络。在CIFAR-10、ImageNet、PASCAL VOC和Penn Treebank上进行的广泛实验，使用卷积模型进行图像分类和语义分割，使用递归模型进行语言建模，表明我们的训练方案产生的有效网络比竞争性修剪方法产生的网络更小，更准确。
We develop an approach to training deep networks while dynamically adjusting their architecture, driven by a principled combination of accuracy and sparsity objectives. Unlike conventional pruning approaches, our method adopts a gradual continuous relaxation of discrete network structure optimization and then samples sparse subnetworks, enabling efficient deep networks to be trained in a growing and pruning manner. Extensive experiments across CIFAR-10, ImageNet, PASCAL VOC, and Penn Treebank, with convolutional models for image classification and semantic segmentation, and recurrent models for language modeling, show that our training scheme yields efficient networks that are smaller and more accurate than those produced by competing pruning methods.