When Does Self-supervision Improve Few-shot Learning?
When Does Self-supervision Improve Few-shot Learning?
复制标题
DOI：
10.1007/978-3-030-58571-6_38
复制
发表时间：
2019-10
期刊：
影响因子：
--
通讯作者：
Jong-Chyi Su;Subhransu Maji;B. Hariharan
中科院分区：
文献类型：
--
作者：
Jong-Chyi Su;Subhransu Maji;B. Hariharan
我们研究了自我监督学习（SSL）在少数学习背景下的作用。虽然最近的研究表明SSL在大型未标记数据集上的好处，但它在小型数据集上的实用性相对未被探索。我们发现，SSL将少量元学习器的相对错误率降低了4%-27%，即使数据集很小并且只利用数据集中的图像。当训练集较小或任务更具挑战性时，改进更大。虽然SSL的好处可能会随着训练集的增加而增加，但我们观察到，当用于元学习和SSL的图像分布不同时，SSL可能会损害性能。我们进行了系统的研究，通过不同程度的领域转移和分析性能的几个元学习者在众多的领域。基于这种分析，我们提出了一种技术，自动选择SSL图像从一个大的，通用的池中的未标记的图像为一个给定的数据集，提供进一步的改进。
We investigate the role of self-supervised learning (SSL) in the context of few-shot learning. Although recent research has shown the benefits of SSL on large unlabeled datasets, its utility on small datasets is relatively unexplored. We find that SSL reduces the relative error rate of few-shot meta-learners by 4%–27%, even when the datasets are small andonlyutilizing images within the datasets. The improvements are greater when the training set is smaller or the task is more challenging. Although the benefits of SSL may increase with larger training sets, we observe that SSL can hurt the performance when the distributions of images used for meta-learning and SSL are different. We conduct a systematic study by varying the degree of domain shift and analyzing the performance of several meta-learners on a multitude of domains. Based on this analysis we present a technique that automatically selects images for SSL from a large, generic pool of unlabeled images for a given dataset that provides further improvements.