Dynamic Activation of Clients and Parameters for Federated Learning over Heterogeneous Graphs
Dynamic Activation of Clients and Parameters for Federated Learning over Heterogeneous Graphs
复制标题
DOI：
10.1109/icde55515.2023.00126
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Zishan Gu;Ke Zhang;Liang Chen;Sun
中科院分区：
文献类型：
--
作者：
Zishan Gu;Ke Zhang;Liang Chen;Sun
在许多真实世界的应用程序中生成的数据可以建模为多类型实体（节点）和关系（链接）的异构图。如今，这些数据通常由分布式客户端生成和存储，使得直接集中式模型训练变得不切实际。虽然每个客户端中的数据都倾向于有偏见的局部分布，但可推广的全局模型仍然经常需要大规模的应用程序。然而，由于客户端之间的通信和同步，大量的客户端强制执行显著的计算开销，而有偏见的本地数据分布表明，并非所有的客户端和参数都应该在所有时间被计算和更新。在这项工作中，我们提出了通过为HGN的联邦训练期间的客户端和参数设计动态激活策略来利用异构图的特性，这是专门设计的关于使用vanilla FedAvg框架训练最先进的异构图神经网络（HGN）的初步研究。此外，我们设计了一个新的解纠缠模型D-HGN，使面向类型的激活模型参数的FedDA。我们所提出的技术的有效性和效率得到了理论和实证分析的支持-我们从理论上分析了FedDA的有效性和收敛性，并从数学上说明了其效率增益;同时，我们证明了FedDA的显着性能增益，并通过基于真实世界异构图合成的多个现实FL设置进行了大量实验，证实了其效率增益。
The data generated in many real-world applications can be modeled as heterogeneous graphs of multi-typed entities (nodes) and relations (links). Nowadays, such data are commonly generated and stored by distributed clients, making direct centralized model training unpractical. While the data in each client are prone to biased local distributions, generalizable global models are still in frequent need for large-scale applications. However, the large number of clients enforce significant computational overhead due to the communication and synchronization among the clients, whereas the biased local data distributions indicate that not all clients and parameters should be computed and updated at all times. Motivated by specifically designed preliminary studies on training a state-of-the-art heterogeneous graph neural network (HGN) with the vanilla FedAvg framework, in this work, we propose to leverage the characteristics of heterogeneous graphs by designing dynamic activation strategies for the clients and parameters during the federated training of HGN, named FedDA. Moreover, we design a novel disentangled model D-HGN to enable type-oriented activation of model parameters for FedDA. The effectiveness and efficiency of our proposed techniques are backed by both theoretical and empirical analysis– We theoretically analyze the validity and convergence of FedDA and mathematically illustrate its efficiency gain; meanwhile, we demonstrate the significant performance gains of FedDA and corroborate its efficiency gains with extensive experiments over multiple realistic FL settings synthesized based on real-world heterogeneous graphs.