Fed2: Feature-Aligned Federated Learning
Fed2: Feature-Aligned Federated Learning
复制标题
DOI：
10.1145/3447548.3467309
复制
发表时间：
2021-08
期刊：
影响因子：
--
通讯作者：
Fuxun Yu;Weishan Zhang;Zhuwei Qin;Zirui Xu;Di Wang;Chenchen Liu;Zhi Tian;Xiang Chen
中科院分区：
文献类型：
--
作者：
Fuxun Yu;Weishan Zhang;Zhuwei Qin;Zirui Xu;Di Wang;Chenchen Liu;Zhi Tian;Xiang Chen
联邦学习通过融合来自本地节点的协作模型来从分散的数据中学习。然而，传统的基于坐标的模型平均FedAvg忽略了每个参数编码的随机信息，并可能遭受结构特征错位。在这项工作中，我们提出了Fed 2，一个功能对齐的联邦学习框架，以解决这个问题，通过建立一个公司的结构，功能对齐的协作模型。Fed 2由两个主要设计组成：首先，我们设计了一个面向特征的模型结构自适应方法，以确保在不同的神经网络结构显式的特征分配。将结构自适应应用于协作模型，可以在非常早期的训练阶段初始化具有相似特征信息的匹配结构。在联邦学习过程中，我们提出了一个特征配对平均方案，以保证对齐的特征分布，并保持在IID或非IID场景下没有特征融合冲突。最终，Fed 2可以在广泛的同质和异构设置下有效地增强联邦学习收敛性能，提供出色的收敛速度，准确性和计算/通信效率。
Federated learning learns from scattered data by fusing collaborative models from local nodes. However, conventional coordinate-based model averaging by FedAvg ignored the random information encoded per parameter and may suffer from structural feature misalignment. In this work, we propose Fed2, a feature-aligned federated learning framework to resolve this issue by establishing a firm structure-feature alignment across the collaborative models. Fed2 is composed of two major designs: First, we design a feature-oriented model structure adaptation method to ensure explicit feature allocation in different neural network structures. Applying the structure adaptation to collaborative models, matchable structures with similar feature information can be initialized at the very early training stage. During the federated learning process, we then propose a feature paired averaging scheme to guarantee aligned feature distribution and maintain no feature fusion conflicts under either IID or non-IID scenarios. Eventually, Fed2 could effectively enhance the federated learning convergence performance under extensive homo- and heterogeneous settings, providing excellent convergence speed, accuracy, and computation/communication efficiency.