AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence
AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence
复制标题
DOI：
10.1109/cvpr46437.2021.01347
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Chengyue Gong;Dilin Wang;Qiang Liu
中科院分区：
文献类型：
--
作者：
Chengyue Gong;Dilin Wang;Qiang Liu
半监督学习（SSL）是一种通过联合利用标记和未标记数据来实现更有效的数据机器学习的关键方法。我们提出了AlphaMatch，一个有效的SSL方法，利用数据增强，通过有效地执行标签之间的一致性的数据点和增强的数据来自他们。我们的关键技术贡献在于：1）使用alpha发散度优先考虑高置信度数据的正则化，实现与FixMatch类似的效果[32]，但以更灵活的方式，以及2）提出一种基于优化的EM算法来加强一致性，其收敛性优于最近SSL方法中使用的迭代正则化过程，如FixMatch，UDA和MixMatch。AlphaMatch简单且易于实现，并且在标准基准测试中始终优于现有技术，例如CIFAR-10，SVHN，CIFAR-100，STL-10。具体来说，我们在CIFAR-10上实现了91.3%的测试准确率，每个类别只有4个标记数据，大大提高了FixMatch之前最好的88.7%的准确率。
Semi-supervised learning (SSL) is a key approach toward more data-efficient machine learning by jointly leverage both labeled and unlabeled data. We propose AlphaMatch, an efficient SSL method that leverages data augmentations, by efficiently enforcing the label consistency between the data points and the augmented data derived from them. Our key technical contribution lies on: 1) using alpha-divergence to prioritize the regularization on data with high confidence, achieving similar effect as FixMatch [32] but in a more flexible fashion, and 2) proposing an optimization-based, EM-like algorithm to enforce the consistency, which enjoys better convergence than iterative regularization procedures used in recent SSL methods such as FixMatch, UDA, and MixMatch. AlphaMatch is simple and easy to implement, and consistently outperforms prior arts on standard benchmarks, e.g. CIFAR-10, SVHN, CIFAR-100, STL-10. Specifically, we achieve 91.3% test accuracy on CIFAR-10 with just 4 labelled data per class, substantially improving over the previously best 88.7% accuracy achieved by FixMatch.