Federated Few-shot Learning
Federated Few-shot Learning
复制标题
DOI：
10.1145/3580305.3599347
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Song Wang;Xingbo Fu;Kaize Ding;Chen Chen-Chen;Huiyuan Chen;Jundong Li
中科院分区：
文献类型：
--
作者：
Song Wang;Xingbo Fu;Kaize Ding;Chen Chen-Chen;Huiyuan Chen;Jundong Li
联邦学习（FL）使多个客户端能够协作学习机器学习模型，而无需交换自己的本地数据。通过这种方式，服务器可以利用所有客户端的计算能力，并在所有客户端的更大数据样本集上训练模型。虽然这种机制在各个领域被证明是有效的，但现有的工作通常假设每个客户都保留足够的培训数据。然而，在实践中，某些客户端只能包含有限数量的样本（即，少量样本）。例如，特定用户使用新的移动设备拍摄的可用照片数据相对较少。在这种情况下，现有的FL工作通常会在这些客户机上遇到显著的性能下降。因此，迫切需要开发一个能够推广到FL场景下数据有限的客户端的few-shot模型。在本文中，我们将这种新问题称为联邦少射学习。然而，由于两个主要原因，这个问题仍然具有挑战性：客户端之间的全球数据差异（即客户端之间数据分布的差异）和每个客户端的本地数据不足（即缺乏足够的本地数据进行培训）。为了克服这两个挑战，我们提出了一种新的联邦少镜头学习框架，该框架具有两个单独更新的模型和专用的训练策略，以减少全局数据方差和局部数据不足的不利影响。在涵盖新闻文章和图像的四个流行数据集上进行了广泛的实验，与最先进的基线相比，验证了我们的框架的有效性。
Federated Learning (FL) enables multiple clients to collaboratively learn a machine learning model without exchanging their own local data. In this way, the server can exploit the computational power of all clients and train the model on a larger set of data samples among all clients. Although such a mechanism is proven to be effective in various fields, existing works generally assume that each client preserves sufficient data for training. In practice, however, certain clients can only contain a limited number of samples (i.e., few-shot samples). For example, the available photo data taken by a specific user with a new mobile device is relatively rare. In this scenario, existing FL efforts typically encounter a significant performance drop on these clients. Therefore, it is urgent to develop a few-shot model that can generalize to clients with limited data under the FL scenario. In this paper, we refer to this novel problem as federated few-shot learning. Nevertheless, the problem remains challenging due to two major reasons: the global data variance among clients (i.e., the difference in data distributions among clients) and the local data insufficiency in each client (i.e., the lack of adequate local data for training). To overcome these two challenges, we propose a novel federated few-shot learning framework with two separately updated models and dedicated training strategies to reduce the adverse impact of global data variance and local data insufficiency. Extensive experiments on four prevalent datasets that cover news articles and images validate the effectiveness of our framework compared with the state-of-the-art baselines.