Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning
Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning
复制标题
DOI：
10.1007/978-3-031-19821-2_2
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Ajinkya Tejankar;Soroush Abbasi Koohpayegani;K. Navaneet;Kossar Pourahmadi;Akshayvarun Subramanya;H. Pirs
中科院分区：
文献类型：
--
作者：
Ajinkya Tejankar;Soroush Abbasi Koohpayegani;K. Navaneet;Kossar Pourahmadi;Akshayvarun Subramanya;H. Pirs
我们对自监督、监督和半监督环境中的表示学习感兴趣。最近的一些自监督学习方法，例如通过将查询图像的嵌入拉近其最近邻居（NN）来聚类图像的均值漂移（MSF）。由于大多数神经网络在设计上都接近查询，因此平均可能不会对查询的嵌入产生太大影响。另一方面，距离较远的神经网络可能与查询在语义上不相关。我们通过使用另一种知识源限制神经网络的搜索空间来推广均值平移的思想，使得神经网络远离查询，同时仍然在语义上相关。我们证明，当约束使用与前一个时期不同的图像增强时，我们的方法（1）在 SSL 设置中优于 MSF；（2）当约束确保 NN 具有与查询相同的伪标签时，在半监督设置中以较少的训练资源优于 PAWS。我们的代码可以在这里找到：https://github.com/UCDvision/CMSF。
We are interested in representation learning in self-supervised, supervised, and semi-supervised settings. Some recent self-supervised learning methods like mean-shift (MSF) cluster images by pulling the embedding of a query image to be closer to its nearest neighbors (NNs). Since most NNs are close to the query by design, the averaging may not affect the embedding of the query much. On the other hand, far away NNs may not be semantically related to the query. We generalize the mean-shift idea by constraining the search space of NNs using another source of knowledge so that NNs are far from the query while still being semantically related. We show that our method (1) outperforms MSF in SSL setting when the constraint utilizes a different augmentation of an image from the previous epoch, and (2) outperforms PAWS in semi-supervised setting with less training resources when the constraint ensures that the NNs have the same pseudo-label as the query. Our code is available here: https://github.com/UCDvision/CMSF.