Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
复制标题
DOI：
10.1109/tpami.2018.2858821
复制
发表时间：
2019-08-01
影响因子：
23.6
通讯作者：
Ishii, Shin
中科院分区：
文献类型：
--
作者：
Miyato, Takeru;Maeda, Shin-Ichi;Ishii, Shin
我们提出了一种新的正则化方法的基础上虚拟对抗损失：一个新的措施，局部光滑的条件标签分布给定的输入。虚拟对抗损失被定义为每个输入数据点周围的条件标签分布对局部扰动的鲁棒性。与对抗训练不同，我们的方法定义了没有标签信息的对抗方向，因此适用于半监督学习。因为我们平滑模型的方向只是“虚拟”对抗，我们称我们的方法为虚拟对抗训练（VAT）。增值税的计算成本相对较低。对于神经网络，虚拟对抗损失的近似梯度可以用不超过两对的前向和反向传播来计算。在我们的实验中，我们将VAT应用于多个基准数据集上的监督和半监督学习任务。通过对基于熵最小化原理的算法进行简单的增强，我们的VAT在SVHN和CIFAR-10上实现了最先进的半监督学习任务。
We propose a new regularization method based on virtual adversarial loss: a new measure of local smoothness of the conditional label distribution given input. Virtual adversarial loss is defined as the robustness of the conditional label distribution around each input data point against local perturbation. Unlike adversarial training, our method defines the adversarial direction without label information and is hence applicable to semi-supervised learning. Because the directions in which we smooth the model are only "virtually" adversarial, we call our method virtual adversarial training (VAT). The computational cost of VAT is relatively low. For neural networks, the approximated gradient of virtual adversarial loss can be computed with no more than two pairs of forward- and back-propagations. In our experiments, we applied VAT to supervised and semi-supervised learning tasks on multiple benchmark datasets. With a simple enhancement of the algorithm based on the entropy minimization principle, our VATachieves state-of-the-art performance for semi-supervised learning tasks on SVHN and CIFAR-10.