Contrastive learning, multi-view redundancy, and linear models
Contrastive learning, multi-view redundancy, and linear models
复制标题
DOI：
--
复制
发表时间：
2020-08
期刊：
影响因子：
--
通讯作者：
Christopher Tosh;A. Krishnamurthy;Daniel J. Hsu
中科院分区：
文献类型：
--
作者：
Christopher Tosh;A. Krishnamurthy;Daniel J. Hsu
自监督学习是一种经验上成功的无监督学习方法，它基于创建人工监督学习问题。一种流行的自我监督表示学习方法是对比学习，它利用自然发生的相似和不相似的数据点对，或相同数据的多个视图。这项工作提供了在多视图环境下对比学习的理论分析，其中每个数据的两个视图是可用的。主要结果是，当两个视图提供关于标签的冗余信息时，学习表征的线性函数在下游预测任务中几乎是最优的。
Self-supervised learning is an empirically successful approach to unsupervised learning based on creating artificial supervised learning problems. A popular self-supervised approach to representation learning is contrastive learning, which leverages naturally occurring pairs of similar and dissimilar data points, or multiple views of the same data. This work provides a theoretical analysis of contrastive learning in the multi-view setting, where two views of each datum are available. The main result is that linear functions of the learned representations are nearly optimal on downstream prediction tasks whenever the two views provide redundant information about the label.