Cross-Silo Federated Learning Across Divergent Domains with Iterative Parameter Alignment
Cross-Silo Federated Learning Across Divergent Domains with Iterative Parameter Alignment
复制标题
DOI：
10.1109/bigdata59044.2023.10386280
复制
发表时间：
2023-11
期刊：
影响因子：
--
通讯作者：
Matt Gorbett;Hossein Shirazi;Indrakshi Ray
中科院分区：
文献类型：
--
作者：
Matt Gorbett;Hossein Shirazi;Indrakshi Ray
从分散在私有资源上的数据的集体知识中学习可以为神经网络提供增强的泛化能力。联邦学习是一种跨远程客户端协作训练机器学习模型的方法，它通过中央服务器的编排组合客户端模型来实现这一目标。然而，当前的方法面临两个关键的限制：i)当客户端域足够不同时，它们难以收敛；ii)当前的聚合技术为每个客户端产生相同的全局模型。在这项工作中，我们通过重新制定典型的联邦学习设置来解决这些问题：我们不是学习单个全局模型，而是学习N个模型，每个模型都针对一个共同目标进行了优化。为了实现这一点，我们对点对点拓扑中共享的模型参数应用加权距离最小化。由此产生的框架，迭代参数对齐，自然适用于跨筒仓设置，并具有以下属性：(i)每个参与者的唯一解决方案，可以选择在联邦中全局收敛每个模型，以及（ii）可选的早期停止机制，以在协作学习设置中引起同伴之间的公平。这些特征共同提供了一个灵活的新框架，用于从在不同数据集上训练的对等模型中迭代学习。我们发现，与最先进的方法相比，该技术在各种数据分区上取得了具有竞争力的结果。此外，我们表明，该方法是鲁棒的分歧领域（即跨节点的不相交类），现有的方法斗争。
Learning from the collective knowledge of data dispersed across private sources can provide neural networks with enhanced generalization capabilities. Federated learning, a method for collaboratively training a machine learning model across remote clients, achieves this by combining client models via the orchestration of a central server. However, current approaches face two critical limitations: i) they struggle to converge when client domains are sufficiently different, and ii) current aggregation techniques produce an identical global model for each client. In this work, we address these issues by reformulating the typical federated learning setup: rather than learning a single global model, we learn N models each optimized for a common objective. To achieve this, we apply a weighted distance minimization to model parameters shared in a peer-to-peer topology. The resulting framework, Iterative Parameter Alignment, applies naturally to the cross-silo setting, and has the following properties: (i) a unique solution for each participant, with the option to globally converge each model in the federation, and (ii) an optional early-stopping mechanism to elicit fairness among peers in collaborative learning settings. These characteristics jointly provide a flexible new framework for iteratively learning from peer models trained on disparate datasets. We find that the technique achieves competitive results on a variety of data partitions compared to state-of-the-art approaches. Further, we show that the method is robust to divergent domains (i.e. disjoint classes across peers) where existing approaches struggle.