Learning a deep convolutional neural network via tensor decomposition
Learning a deep convolutional neural network via tensor decomposition
复制标题
DOI：
10.1093/imaiai/iaaa042
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
Samet Oymak;M. Soltanolkotabi
中科院分区：
文献类型：
--
作者：
Samet Oymak;M. Soltanolkotabi
本文研究了深度卷积神经网络的权值学习问题。我们考虑一个网络，其中卷积在非重叠的补丁上进行。我们开发了一种从训练数据中同时学习所有核的算法。我们的方法被称为深度张量分解（DeepTD）是基于低秩张量分解。在训练数据的可实现模型下，我们从理论上研究了DeepTD，该模型从高斯分布中选择输入，并根据种植的卷积核生成标签。我们证明了DeepTD是样本效率高的，并且只要样本大小超过网络中卷积权重的总数，就可以证明它是有效的。
In this paper, we study the problem of learning the weights of a deep convolutional neural network. We consider a network where convolutions are carried out over non-overlapping patches. We develop an algorithm for simultaneously learning all the kernels from the training data. Our approach dubbed deep tensor decomposition (DeepTD) is based on a low-rank tensor decomposition. We theoretically investigate DeepTD under a realizable model for the training data where the inputs are chosen i.i.d. from a Gaussian distribution and the labels are generated according to planted convolutional kernels. We show that DeepTD is sample efficient and provably works as soon as the sample size exceeds the total number of convolutional weights in the network.