Demystifying Dropout
Demystifying Dropout
复制标题
DOI：
--
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
Hongchang Gao;J. Pei;Heng Huang
中科院分区：
文献类型：
--
作者：
Hongchang Gao;J. Pei;Heng Huang
Dropout是一种用于训练大规模深度神经网络以缓解过度拟合问题的流行技术。为了揭示其获得成功的深层原因，许多著作都试图从不同的角度进行解释。在本文中，不同于现有的作品，我们从一个新的角度来探讨它提供了新的见解，这条线的研究。详细地，我们解开了dropout的向前和向后传递。然后，我们发现这两个通道需要不同程度的噪声来提高深度神经网络的泛化性能。基于这一观察，我们提出了增强dropout，它采用不同的丢弃策略，在向前和向后通过，以改善标准dropout。实验结果验证了艾德提出的方法的有效性。
Dropout is a popular technique to train large-scale deep neural networks to alleviate the overfitting problem. To disclose the underlying reason for its gain, numerous works have tried to explain it from different perspectives. In this paper, unlike existing works, we explore it from a new perspective to provide new insight into this line of research. In detail, we disentangle the forward and backward pass of dropout. Then, we find that these two passes need different levels of noise to improve the generalization performance of deep neural networks. Based on this observation, we propose the augmented dropout, which employs different dropping strategies in the forward and backward pass, to improve the standard dropout. Experimental results have verified the effectiveness of our proposed method.