Knowledge-Aided Federated Learning for Energy-Limited Wireless Networks
Knowledge-Aided Federated Learning for Energy-Limited Wireless Networks
复制标题
DOI：
10.1109/tcomm.2023.3261383
复制
发表时间：
2022-09
影响因子：
8.3
通讯作者：
Zhixiong Chen;Wenqiang Yi;Yuanwei Liu;A. Nallanathan
中科院分区：
文献类型：
--
作者：
Zhixiong Chen;Wenqiang Yi;Yuanwei Liu;A. Nallanathan
传统的基于模型聚合的联邦学习（FL）方法要求所有局部模型具有相同的架构，这无法支持异构局部模型的实际场景。此外，对于资源有限的无线网络来说，频繁的模型交换成本高昂，因为现代深度神经网络通常具有超过一百万个参数。为了应对这些挑战，我们首先提出了一种新颖的知识辅助 FL (KFL) 框架，该框架在每轮学习过程中聚合轻量级数据特征，即知识。该框架允许设备独立设计机器学习模型，并减少训练过程中的通信开销。然后，我们从理论上分析了所提出的框架在非凸损失函数设置下的收敛界限，表明在每轮中调度更多的数据量有助于提高学习性能。另外，如果整个学习过程中的总调度数据量是固定的，则应在早期轮次调度大数据量。受此启发，我们定义了一个新的目标函数，即加权调度数据样本量，将不明确的全局损失最小化问题转化为易于处理的设备调度、带宽分配和功率控制问题。为了处理未知的时变无线信道，我们借助 Lyapunov 优化框架将每轮所考虑的问题转化为确定性问题。然后，我们通过凸优化技术得出最优的带宽分配和功率控制解决方案。我们还开发了一种高效的在线设备调度算法，以实现学习过程中的能量学习权衡。在高度异构的本地数据分布下的两个典型数据集（即 MNIST 和 CIFAR-10）上的实验结果表明，与传统的基于模型聚合的算法相比，所提出的 KFL 能够减少超过 99% 的通信开销，同时获得更好的学习性能。此外，所提出的设备调度算法比基准调度方案收敛得更快。
The conventional model aggregation-based federated learning (FL) approach requires all local models to have the same architecture, which fails to support practical scenarios with heterogeneous local models. Moreover, the frequent model exchange is costly for resource-limited wireless networks since modern deep neural networks usually have over a million parameters. To tackle these challenges, we first propose a novel knowledge-aided FL (KFL) framework, which aggregates light high-level data features, namely knowledge, in the per-round learning process. This framework allows devices to design their machine-learning models independently and reduces the communication overhead in the training process. We then theoretically analyze the convergence bound of the proposed framework under a non-convex loss function setting, revealing that scheduling more data volume in each round helps to improve the learning performance. In addition, large data volume should be scheduled in early rounds if the total scheduled data volume during the entire learning course is fixed. Inspired by this, we define a new objective function, i.e., the weighted scheduled data sample volume, to transform the inexplicit global loss minimization problem into a tractable one for device scheduling, bandwidth allocation, and power control. To deal with unknown time-varying wireless channels, we transform the considered problem into a deterministic problem for each round with the assistance of the Lyapunov optimization framework. Then, we derive the optimal bandwidth allocation and power control solution by convex optimization techniques. We also develop an efficient online device scheduling algorithm to achieve an energy-learning trade-off in the learning process. Experimental results on two typical datasets (i.e., MNIST and CIFAR-10) under highly heterogeneous local data distributions show that the proposed KFL is capable of reducing over 99% communication overhead while achieving better learning performance than the conventional model aggregation-based algorithms. In addition, the proposed device scheduling algorithm converges faster than the benchmark scheduling schemes.