Over-Sampling in a Deep Neural Network
Over-Sampling in a Deep Neural Network
复制标题
DOI：
--
复制
发表时间：
2015-02
期刊：
影响因子：
--
通讯作者：
Andrew J. R. Simpson
中科院分区：
文献类型：
--
作者：
Andrew J. R. Simpson
深度神经网络（DNN）是计算机视觉和听觉等许多工程问题的最新技术。深度神经网络成功的一个关键因素是可扩展性——更大的网络工作得更好。然而，这种可伸缩性的原因还没有得到很好的理解。在这里，我们将深度神经网络解释为一个离散系统，线性滤波器之后是非线性激活，服从采样理论定律。在这种情况下，我们证明了过采样网络更具选择性，学习速度更快，学习更稳健。我们的发现可能最终推广到人类大脑。
Deep neural networks (DNN) are the state of the art on many engineering problems such as computer vision and audition. A key factor in the success of the DNN is scalability - bigger networks work better. However, the reason for this scalability is not yet well understood. Here, we interpret the DNN as a discrete system, of linear filters followed by nonlinear activations, that is subject to the laws of sampling theory. In this context, we demonstrate that over-sampled networks are more selective, learn faster and learn more robustly. Our findings may ultimately generalize to the human brain.