Improving Neural Networks with Dropout
Improving Neural Networks with Dropout
复制标题
DOI：
--
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
Nitish Srivastava
中科院分区：
文献类型：
--
作者：
Nitish Srivastava
Nitish Srivastava多伦多大学计算机科学研究生院理学硕士2013年拥有大量参数的深度神经网络是非常强大的机器学习系统。然而，过拟合是此类网络中的一个严重问题。大型网络使用起来也很慢，这使得在测试时很难通过组合许多不同的大型神经网络来处理过拟合。Dropout是解决这个问题的一种技术。其关键思想是在训练过程中从神经网络中随机丢弃单元（沿着它们的连接）。这可以防止单位过度适应。丢弃单元会在训练期间创建稀疏网络。可能的稀疏网络的数量是网络中单元数量的指数。在测试时，所有可能的细化网络结合使用近似模型平均程序。这种近似模型组合之后的丢弃训练显着减少了过度拟合，并对其他正则化方法进行了重大改进。在这项工作中，我们描述了使用dropout提高神经网络性能的模型，通常在基准数据集上获得最先进的结果。
Improving Neural Networks with Dropout Nitish Srivastava Master of Science Graduate Department of Computer Science University of Toronto 2013 Deep neural nets with a huge number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from a neural network during training. This prevents the units from co-adapting too much. Dropping units creates thinned networks during training. The number of possible thinned networks is exponential in the number of units in the network. At test time all possible thinned networks are combined using an approximate model averaging procedure. Dropout training followed by this approximate model combination significantly reduces overfitting and gives major improvements over other regularization methods. In this work, we describe models that improve the performance of neural networks using dropout, often obtaining state-of-the-art results on benchmark datasets.