SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training
SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training
复制标题
DOI：
--
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Enmao Diao;Jie Ding;V. Tarokh
中科院分区：
文献类型：
--
作者：
Enmao Diao;Jie Ding;V. Tarokh
联邦学习允许通过使用许多分布式客户机的计算和私有数据资源来训练机器学习模型。关于联邦学习（FL）的大多数现有结果都假设客户端具有真值标签。然而，在许多实际场景中，由于缺乏专业知识或资源，客户端可能无法标记特定于任务的数据。我们提出半监督学习（semi - ifl）来解决将通信高效的FL（如fedag）与半监督学习（SSL）相结合的问题。在SemiFL中，客户端拥有完全未标记的数据，可以训练多个局部epoch以降低通信成本，而服务器拥有少量标记数据。我们对基于数据增强的SSL方法的成功提供了理论理解，以说明通信高效FL与SSL的普通组合的瓶颈。为了解决这个问题，我们提出了“使用标记数据微调全局模型”和“使用全局模型生成伪标签”的替代训练。我们进行了大量的实验，并证明我们的方法显着提高了带有多个局部epoch的未标记客户端训练的标记服务器的性能。此外，我们的方法优于许多现有的SSFL基线，并且与最先进的FL和SSL结果相比具有竞争力。
Federated Learning allows the training of machine learning models by using the computation and private data resources of many distributed clients. Most existing results on Federated Learning (FL) assume the clients have ground-truth labels. However, in many practical scenarios, clients may be unable to label task-specific data due to a lack of expertise or resource. We propose SemiFL to address the problem of combining communication-efficient FL such as FedAvg with Semi-Supervised Learning (SSL). In SemiFL, clients have completely unlabeled data and can train multiple local epochs to reduce communication costs, while the server has a small amount of labeled data. We provide a theoretical understanding of the success of data augmentation-based SSL methods to illustrate the bottleneck of a vanilla combination of communication-efficient FL with SSL. To address this issue, we propose alternate training to `fine-tune global model with labeled data' and `generate pseudo-labels with the global model.' We conduct extensive experiments and demonstrate that our approach significantly improves the performance of a labeled server with unlabeled clients training with multiple local epochs. Moreover, our method outperforms many existing SSFL baselines and performs competitively with the state-of-the-art FL and SSL results.