Heterogeneity-Aware Adaptive Federated Learning Scheduling
Heterogeneity-Aware Adaptive Federated Learning Scheduling
复制标题
DOI：
10.1109/bigdata55660.2022.10020721
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Jingoo Han;Ahmad Faraz Khan;Syed Zawad;A. Anwar;Nathalie Baracaldo Angel;Yi Zhou;Feng Yan;A. Butt
中科院分区：
文献类型：
--
作者：
Jingoo Han;Ahmad Faraz Khan;Syed Zawad;A. Anwar;Nathalie Baracaldo Angel;Yi Zhou;Feng Yan;A. Butt
联邦学习（FL）正在成为一种重要的分布式机器学习方法，它考虑了隐私和安全问题，同时使用本地化数据在各种客户端之间训练共享模型。FL中的关键挑战之一是硬件资源和本地数据集的异构性，这是由于合并不同客户端的性质。由于资源的异质性，参与客户端的可用性是不稳定的，随着时间的推移，他们的资源使用模式变得动态。这导致资源浪费和落伍问题。由于数据异构性，引入了额外的挑战，导致模型偏差和模型性能差。然而，大多数现有的FL系统不适合异构环境，因为这些方法不适应各种动态变化的资源使用模式和训练过程中的精度趋势。为此，我们提出了一个异构性感知调度，这是自适应的准确性趋势和各种资源使用模式。我们提出的调度提供了不同的调度旋钮，以实现不同的目标，如资源有效的快速训练，资源公平性，准确性公平性和高模型性能。据我们所知，这是第一次努力减轻资源和数据异构性的影响，同时提供基于动态变化的资源使用模式和准确性趋势的自适应调度。
Federated learning (FL) is becoming an important distributed machine learning approach that considers privacy and security concerns while training a shared model across various clients with localized data. One of the key challenges in FL is heterogeneity in both hardware resources and local datasets due to the nature of incorporating diverse clients. Given the resource heterogeneity, the availability of participating clients is not stable over time and their resource usage patterns become dynamic. This leads to resource wastage and straggler issues. Additional challenges are introduced due to data heterogeneity, causing model biasness and poor model performance. However, most existing FL systems are not well suited to heterogeneous environments because those approaches are not adaptive to various and dynamically changing resource usage patterns and accuracy trends during training process. To this end, we propose a heterogeneity-aware scheduling which is adaptive to the accuracy trends and various resource usage patterns. Our proposed scheduling provides different scheduling knobs for achieving different goals such as resource-efficient fast training, resource fairness, accuracy fairness, and high model performance. To the best of our knowledge, this is the first effort to mitigate effects of resource and data heterogeneity while providing adaptive scheduling based on dynamically changing resource usage patterns and accuracy trends.