An Optimal Transport Approach to Personalized Federated Learning
An Optimal Transport Approach to Personalized Federated Learning
复制标题
DOI：
10.1109/jsait.2022.3182355
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Farzan Farnia;Amirhossein Reisizadeh;Ramtin Pedarsani;A. Jadbabaie
中科院分区：
文献类型：
--
作者：
Farzan Farnia;Amirhossein Reisizadeh;Ramtin Pedarsani;A. Jadbabaie
联邦学习是一种分布式机器学习范例，旨在使用许多分布式客户端的本地数据来训练模型。联邦学习的一个关键挑战是客户端之间的数据样本可能分布不一样。为了应对这一挑战，提出了个性化联合学习，其目标是根据每个客户的数据分布定制学习模型。在本文中，我们关注这个问题，提出了一种基于最优传输（FedOT）的新颖的个性化联邦学习方案作为学习算法，该算法学习将数据点转移到公共分布的最优传输图以及所应用传输图下的预测模型。为了制定 FedOT 问题，我们将两个概率分布之间的标准最优传输任务扩展到多边缘最优传输问题，其目标是将样本从多个分布传输到公共概率域。然后，我们利用多边际最优运输问题的结果将 FedOT 表述为最小-最大优化问题，并分析其泛化和优化特性。我们讨论了几个数值实验的结果，以评估联邦学习问题中异构数据分布下 FedOT 的性能。
Federated learning is a distributed machine learning paradigm, which aims to train a model using the local data of many distributed clients. A key challenge in federated learning is that the data samples across the clients may not be identically distributed. To address this challenge, personalized federated learning with the goal of tailoring the learned model to the data distribution of every individual client has been proposed. In this paper, we focus on this problem and propose a novel personalized Federated Learning scheme based on Optimal Transport (FedOT) as a learning algorithm that learns the optimal transport maps for transferring data points to a common distribution as well as the prediction model under the applied transport map. To formulate the FedOT problem, we extend the standard optimal transport task between two probability distributions to multi-marginal optimal transport problems with the goal of transporting samples from multiple distributions to a common probability domain. We then leverage the results on multi-marginal optimal transport problems to formulate FedOT as a min-max optimization problem and analyze its generalization and optimization properties. We discuss the results of several numerical experiments to evaluate the performance of FedOT under heterogeneous data distributions in federated learning problems.