Learning to Share: simultaneous parameter tying and Sparsification in Deep Learning
Learning to Share: simultaneous parameter tying and Sparsification in Deep Learning
复制标题
DOI：
--
复制
发表时间：
2018-02
期刊：
影响因子：
--
通讯作者：
Dejiao Zhang;Haozhu Wang;Mário A. T. Figueiredo;L. Balzano
中科院分区：
文献类型：
--
作者：
Dejiao Zhang;Haozhu Wang;Mário A. T. Figueiredo;L. Balzano
·动机：DNN通常需要昂贵的存储和计算。·目标：通过(I)同时删除不重要的神经元；(Ii)将对应于强相关神经元的权重捆绑在一起来压缩DNN。·结果：通过自动将高度相关特征对应的权重捆绑在一起，我们减轻了噪声输入或共同适应可能导致的强相关性的负面影响。
• Motivation: DNNs usually require expensive storage and computation. • Goal: compress DNNs by (i) simultaneously eliminating unimportant neurons; (ii) tying together weights that correspond to strongly correlated neurons. • Outcome: by automatically tying together weights corresponding to highly correlated features, we alleviate the negative effect of strong correlations that may be induced by noisy inputs or co-adaption.