Provable Methods for Training Neural Networks with Sparse Connectivity
Provable Methods for Training Neural Networks with Sparse Connectivity
复制标题
训练具有稀疏连接的神经网络的可证明方法
DOI：
--
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
Anima Anandkumar
中科院分区：
文献类型：
--
作者：
Hanie Sedghi;Anima Anandkumar
我们提供新颖的有保证的方法来训练具有稀疏连接的前馈神经网络。我们利用之前开发的用于学习线性网络的技术，并表明它们也可以有效地用于学习非线性网络。我们对涉及标签和输入得分函数的矩进行操作，并表明它们的分解可在温和条件下产生深度网络第一层的权重矩阵。在实践中，我们方法的输出可以用作梯度下降的有效初始化器。
We provide novel guaranteed approaches for training feedforward neural networks with sparse connectivity. We leverage on the techniques developed previously for learning linear networks and show that they can also be effectively adopted to learn non-linear networks. We operate on the moments involving label and the score function of the input, and show that their factorization provably yields the weight matrix of the first layer of a deep network under mild conditions. In practice, the output of our method can be employed as effective initializers for gradient descent.