CircConv: A Structured Convolution with Low Complexity
CircConv: A Structured Convolution with Low Complexity
复制标题
DOI：
10.1609/aaai.v33i01.33014287
复制
发表时间：
2019-02
期刊：
影响因子：
--
通讯作者：
Siyu Liao;Zhe Li;Liang Zhao;Qinru Qiu;Yanzhi Wang;Bo Yuan
中科院分区：
文献类型：
--
作者：
Siyu Liao;Zhe Li;Liang Zhao;Qinru Qiu;Yanzhi Wang;Bo Yuan
深度神经网络（DNN），特别是深度卷积神经网络（CNN），已经成为各种机器学习应用中的强大技术。然而，DNN的大模型尺寸对计算资源和权重存储产生高要求，从而限制了DNN的实际部署。为了克服这些限制，本文提出将循环结构强加给卷积层的构造，从而产生循环卷积层（CircConvs）和循环CNN。循环结构和模型可以从头开始训练，也可以从预训练的非循环模型重新训练，从而使其非常灵活地用于不同的训练环境。通过大量的实验，这种强大的structureimposing方法被证明能够大大减少卷积层的参数数量，并通过使用循环张量的快速乘法来显著节省计算成本。
Deep neural networks (DNNs), especially deep convolutional neural networks (CNNs), have emerged as the powerful technique in various machine learning applications. However, the large model sizes of DNNs yield high demands on computation resource and weight storage, thereby limiting the practical deployment of DNNs. To overcome these limitations, this paper proposes to impose the circulant structure to the construction of convolutional layers, and hence leads to circulant convolutional layers (CircConvs) and circulant CNNs. The circulant structure and models can be either trained from scratch or re-trained from a pre-trained non-circulant model, thereby making it very flexible for different training environments. Through extensive experiments, such strong structureimposing approach is proved to be able to substantially reduce the number of parameters of convolutional layers and enable significant saving of computational cost by using fast multiplication of the circulant tensor.