Do Deep Nets Really Need to be Deep?
Do Deep Nets Really Need to be Deep?
复制标题
DOI：
--
复制
发表时间：
2013-12
期刊：
影响因子：
--
通讯作者：
Jimmy Ba;R. Caruana
中科院分区：
文献类型：
--
作者：
Jimmy Ba;R. Caruana
目前，深度神经网络在语音识别和计算机视觉等问题上是最先进的。在本文中，我们通过实验证明，浅层前馈网络可以学习以前由深层网络学习的复杂函数，并达到以前只能通过深层模型才能达到的精度。此外，在某些情况下，浅层网络可以使用与原始深层模型相同数量的参数来学习这些深层函数。在Timit音素识别和CIFAR-10图像识别任务中，可以训练浅层网络，其性能类似于复杂的、精心设计的、更深的卷积模型。
Currently, deep neural networks are the state of the art on problems such as speech recognition and computer vision. In this paper we empirically demonstrate that shallow feed-forward nets can learn the complex functions previously learned by deep nets and achieve accuracies previously only achievable with deep models. Moreover, in some cases the shallow nets can learn these deep functions using the same number of parameters as the original deep models. On the TIMIT phoneme recognition and CIFAR-10 image recognition tasks, shallow nets can be trained that perform similarly to complex, well-engineered, deeper convolutional models.