Decentralized Unsupervised Learning of Visual Representations
Decentralized Unsupervised Learning of Visual Representations
复制标题
DOI：
10.24963/ijcai.2022/323
复制
发表时间：
2021-11
期刊：
影响因子：
--
通讯作者：
Yawen Wu;Zhepeng Wang;Dewen Zeng;Meng Li;Yiyu Shi;Jingtong Hu
中科院分区：
文献类型：
--
作者：
Yawen Wu;Zhepeng Wang;Dewen Zeng;Meng Li;Yiyu Shi;Jingtong Hu
协作学习使分布式客户端能够学习用于预测的共享模型，同时将训练数据保持在每个客户端上。然而，现有的协作学习方法需要完全标记的数据进行训练，这是不方便或有时不可行的，由于高标记成本和专业知识的要求。缺乏标签使得协作学习在许多现实环境中不切实际。自监督学习可以通过从未标记的数据中学习来解决这一挑战。对比学习（CL）是一种自监督学习方法，可以有效地从未标记的图像数据中学习视觉表示。然而，在客户端上收集的分布式数据通常在客户端之间不是独立和相同分布的（非IID），并且每个客户端可能只有少数几类数据，这降低了CL和学习表示的性能。为了解决这个问题，我们提出了一个协作对比学习框架，包括两种方法：特征融合和邻域匹配，通过这种方法，在客户端之间学习一个统一的特征空间，以获得更好的数据表示。特征融合为每个客户端提供远程特征作为准确的对比信息，以更好地进行本地学习。邻域匹配进一步将每个客户端的本地特征与远程特征对齐，使得可以学习客户端之间的良好聚类的特征。大量的实验表明了该框架的有效性。它在IID数据上的性能比其他方法高出11%，并与集中式学习的性能相匹配。
Collaborative learning enables distributed clients to learn a shared model for prediction while keeping the training data local on each client. However, existing collaborative learning methods require fully-labeled data for training, which is inconvenient or sometimes infeasible to obtain due to the high labeling cost and the requirement of expertise. The lack of labels makes collaborative learning impractical in many realistic settings. Self-supervised learning can address this challenge by learning from unlabeled data. Contrastive learning (CL), a self-supervised learning approach, can effectively learn visual representations from unlabeled image data. However, the distributed data collected on clients are usually not independent and identically distributed (non-IID) among clients, and each client may only have few classes of data, which degrades the performance of CL and learned representations. To tackle this problem, we propose a collaborative contrastive learning framework consisting of two approaches: feature fusion and neighborhood matching, by which a unified feature space among clients is learned for better data representations. Feature fusion provides remote features as accurate contrastive information to each client for better local learning. Neighborhood matching further aligns each client’s local features to the remote features such that well-clustered features among clients can be learned. Extensive experiments show the effectiveness of the proposed framework. It outperforms other methods by 11% on IID data and matches the performance of centralized learning.