Spectral Norm Regularization for Improving the Generalizability of Deep Learning
Spectral Norm Regularization for Improving the Generalizability of Deep Learning
复制标题
DOI：
--
复制
发表时间：
2017-05
期刊：
影响因子：
--
通讯作者：
Yuichi Yoshida;Takeru Miyato
中科院分区：
文献类型：
--
作者：
Yuichi Yoshida;Takeru Miyato
我们基于对输入扰动的敏感性来研究深度学习的可推广性。我们假设数据对扰动的高度敏感性降低了神经网络的性能，为了降低数据对扰动的敏感性，我们提出了一种简单有效的正则化方法，称为谱范数正则化，它惩罚神经网络中权矩阵的高谱范数。我们通过实验证实，使用谱范数正则化训练的模型比其他基线方法具有更好的泛化能力，从而为上述假设提供了支持性证据。
We investigate the generalizability of deep learning based on the sensitivity to input perturbation. We hypothesize that the high sensitivity to the perturbation of data degrades the performance on it. To reduce the sensitivity to perturbation, we propose a simple and effective regularization method, referred to as spectral norm regularization, which penalizes the high spectral norm of weight matrices in neural networks. We provide supportive evidence for the abovementioned hypothesis by experimentally confirming that the models trained using spectral norm regularization exhibit better generalizability than other baseline methods.