Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives
Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives
复制标题
DOI：
10.1609/aaai.v36i1.19972
复制
发表时间：
2022-01
期刊：
影响因子：
--
通讯作者：
David T. Hoffmann;Nadine Behrmann;Juergen Gall;T. Brox;M. Noroozi
中科院分区：
文献类型：
--
作者：
David T. Hoffmann;Nadine Behrmann;Juergen Gall;T. Brox;M. Noroozi
本文介绍了排名信息噪声对比估计（RINCE），一个新的成员在家庭的InfoNCE损失，保持了排名排序的正样本。与标准的InfoNCE损失相比，它需要将训练对严格地二进制分离为相似和不相似的样本，RINCE可以利用有关相似性排名的信息来学习相应的嵌入空间。我们表明，建议的损失函数学习有利的嵌入相比，标准的InfoNCE，只要至少有噪音的排名信息可以获得或当定义的积极和消极的是模糊的。我们证明了这一点的监督分类任务，额外的超类标签和嘈杂的相似性分数。此外，我们还通过对视频进行无监督表示学习的实验，证明了RINCE也可以应用于无监督训练。特别是，嵌入产生更高的分类精度，检索率和执行更好的分布外检测比标准的InfoNCE损失。
This paper introduces Ranking Info Noise Contrastive Estimation (RINCE), a new member in the family of InfoNCE losses that preserves a ranked ordering of positive samples. In contrast to the standard InfoNCE loss, which requires a strict binary separation of the training pairs into similar and dissimilar samples, RINCE can exploit information about a similarity ranking for learning a corresponding embedding space. We show that the proposed loss function learns favorable embeddings compared to the standard InfoNCE whenever at least noisy ranking information can be obtained or when the definition of positives and negatives is blurry. We demonstrate this for a supervised classification task with additional superclass labels and noisy similarity scores. Furthermore, we show that RINCE can also be applied to unsupervised training with experiments on unsupervised representation learning from videos. In particular, the embedding yields higher classification accuracy, retrieval rates and performs better on out-of-distribution detection than the standard InfoNCE loss.