Optimizing Aggregation Frequency for Hierarchical Model Training in Heterogeneous Edge Computing
Optimizing Aggregation Frequency for Hierarchical Model Training in Heterogeneous Edge Computing
复制标题
异构边缘计算中分层模型训练的聚合频率优化
DOI：
10.1109/tmc.2022.3149584
复制
发表时间：
2023-07
影响因子：
7.9
通讯作者：
Lei Yang;Yingqi Gan;Jiannong Cao;Zhenyu Wang
中科院分区：
文献类型：
--
作者：
Lei Yang;Yingqi Gan;Jiannong Cao;Zhenyu Wang
联邦学习（FL）已被广泛应用于边缘计算中的分布式机器学习。在FL中，模型参数从客户端迭代聚合到中心服务器，这容易成为通信瓶颈和单点故障。为了解决这些问题，分层模型训练框架如分层联邦学习（HFL）和E-Tree学习被提出。分层模型训练框架中最具挑战性的问题之一是优化不同层次边缘设备的聚合频率。因为，在边缘计算环境中，资源的异构性可能会引入由于等待慢速工作者而导致的同步延迟，并严重影响训练性能。本文解决了弱同步的问题，即同一级别的边缘设备在本地更新和/或模型聚合时具有不同的频率。现有的基于弱同步的工作缺乏定量确定各边缘器件聚合频率的解决方案。因此，我们提出了一种基于资源的聚合频率控制方法，称为RAF，该方法根据异构资源确定边缘设备的最优聚合频率，以最小化损失函数。本文提出的方法可以减少等待时间，充分利用边缘设备的资源。此外，RAF在模型训练过程中动态调整不同阶段的聚合频率，以达到快速收敛和高精度。我们通过在我们自己开发的边缘计算测试平台上使用真实数据集的广泛实验来评估RAF的性能。评估结果表明，RAF在学习精度和收敛速度方面优于基准方法。
Federated Learning (FL) has been widely used for distributed machine learning in edge computing. In FL, the model parameters are iteratively aggregated from the clients to a central server, which is inclined to be the communication bottleneck and single point of failure. To solve these drawbacks, hierarchical model training frameworks like Hierarchical Federated Learning (HFL) and E-Tree learning have been proposed. One of the most challenging problems in the hierarchical model training framework is optimizing the aggregation frequencies of the edge devices at various levels. Because, in an edge computing environment, heterogeneity in the resource can introduce synchronization delays caused by waiting for slow workers and significantly impact the training performance. This paper tackles the problem with weak synchronization where edge devices on the same level have different frequencies on local updates and/or model aggregations. Existing works based on weak synchronization lack solutions to quantitatively determine the aggregation frequencies of each edge device. Thus, we propose a resource-based aggregation frequency controlling method, termed RAF, which determines the optimal aggregation frequencies of edge devices to minimize the loss function according to heterogeneous resources. Our proposed method can alleviate the waiting time and fully utilize the resources of the edge devices. Besides, RAF dynamically adjusts the aggregation frequencies at different phases during the model training to achieve fast convergence speed and high accuracy. We evaluated the performance of RAF via extensive experiments with real datasets on our self-developed edge computing testbed. Evaluation results demonstrate that RAF outperforms the benchmark approaches in terms of learning accuracy and convergence speed.