WCP: Worst-Case Perturbations for Semi-Supervised Deep Learning
WCP: Worst-Case Perturbations for Semi-Supervised Deep Learning
复制标题
DOI：
10.1109/cvpr42600.2020.00397
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Liheng Zhang;Guo-Jun Qi
中科院分区：
文献类型：
--
作者：
Liheng Zhang;Guo-Jun Qi
在本文中，我们提出了一种新的正则化机制，通过最小化{\em最坏情况摄动}（WCP）来训练深度网络。它基于鲁棒模型最不可能受到小扰动影响的想法，因此它的输出决策在标记和未标记的示例上都应该尽可能稳定。我们将考虑两种形式的WCP正则化——加性和DropConnect扰动，它们分别对网络权重施加加性噪声，并通过放弃网络连接来进行结构改变。我们将证明，通过用谱方法求解各自的优化问题，可以推导出这两种扰动的最坏情况。WCP可以在标记和未标记的数据上最小化，这样网络就可以以半监督的方式进行训练。这导致了一种新的半监督分类器范例，通过在网络权重和结构上施加的最坏情况扰动存在的情况下稳定预测输出。
In this paper, we present a novel regularization mechanism for training deep networks by minimizing the {\em Worse-Case Perturbation} (WCP). It is based on the idea that a robust model is least likely to be affected by small perturbations, such that its output decisions should be as stable as possible on both labeled and unlabeled examples. We will consider two forms of WCP regularizations -- additive and DropConnect perturbations, which impose additive noises on network weights, and make structural changes by dropping the network connections, respectively. We will show that the worse cases of both perturbations can be derived by solving respective optimization problems with spectral methods. The WCP can be minimized on both labeled and unlabeled data so that networks can be trained in a semi-supervised fashion. This leads to a novel paradigm of semi-supervised classifiers by stabilizing the predicted outputs in presence of the worse-case perturbations imposed on the network weights and structures.