Max-Margin Contrastive Learning
Max-Margin Contrastive Learning
复制标题
DOI：
10.1609/aaai.v36i8.20796
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Anshul B. Shah;S. Sra;Ramalingam Chellappa;A. Cherian
中科院分区：
文献类型：
--
作者：
Anshul B. Shah;S. Sra;Ramalingam Chellappa;A. Cherian
标准的对比学习方法通常需要大量的否定来进行有效的无监督学习，并且通常表现出缓慢的收敛。我们怀疑这种行为是由于用于提供阳性对比度的阴性的次优选择。我们从支持向量机（SVM）中获得灵感，提出了最大边缘对比学习（MMCL），从而克服了这一困难。我们的方法选择消极的稀疏支持向量，通过二次优化问题获得的，和对比性是通过最大化的决策余量。由于SVM优化在计算上要求很高，特别是在端到端的设置中，我们提出了减轻计算负担的简化方法。我们在标准视觉基准数据集上验证了我们的方法，在最先进的无监督表示学习中表现出更好的性能，同时具有更好的经验收敛特性。
Standard contrastive learning approaches usually require a large number of negatives for effective unsupervised learning and often exhibit slow convergence. We suspect this behavior is due to the suboptimal selection of negatives used for offering contrast to the positives. We counter this difficulty by taking inspiration from support vector machines (SVMs) to present max-margin contrastive learning (MMCL). Our approach selects negatives as the sparse support vectors obtained via a quadratic optimization problem, and contrastiveness is enforced by maximizing the decision margin. As SVM optimization can be computationally demanding, especially in an end-to-end setting, we present simplifications that alleviate the computational burden. We validate our approach on standard vision benchmark datasets, demonstrating better performance in unsupervised representation learning over state-of-the-art, while having better empirical convergence properties.