FLOAT: Federated Learning Optimizations with Automated Tuning
FLOAT: Federated Learning Optimizations with Automated Tuning
复制标题
DOI：
10.1145/3627703.3650081
复制
发表时间：
2024-04
期刊：
影响因子：
--
通讯作者：
Ahmad Faraz Khan;A. Khan;A. Abdelmoniem;Samuel Fountain;Ali R. Butt;Ali Anwar
中科院分区：
文献类型：
--
作者：
Ahmad Faraz Khan;A. Khan;A. Abdelmoniem;Samuel Fountain;Ali R. Butt;Ali Anwar
联合学习（FL）已成为一种强大的方法，可以在不需要数据共享的情况下实现协作分布式模型训练。然而，FL与固有的异质性挑战，导致的问题，如掉队，辍学，和性能变化。选择运行FL实例的客户端至关重要，但现有策略引入了偏见和参与问题，并且没有考虑资源效率。由于系统资源的动态性质，为增加客户参与而提出的通信和培训加速解决方案也达不到要求。我们在本文中解决这些挑战，设计FLOAT，一个新的框架，旨在提高FL客户端资源意识。FLOAT动态优化资源利用率，以满足培训期限，并通过各种优化技术减少掉队和辍学，从而增强模型收敛性和提高性能。FLOAT利用多目标人工反馈强化学习（RLHF）来自动选择优化技术及其配置，并根据客户端的资源条件进行定制。此外，FLOAT无缝集成到现有的FL系统中，保持异步和同步FL设置的非侵入性和多功能性。根据我们的评估，FLOAT将准确性提高了53%，将客户端丢失率降低了78倍，并将通信，计算和内存利用率分别提高了81倍，44倍和20倍。
Federated Learning (FL) has emerged as a powerful approach that enables collaborative distributed model training without the need for data sharing. However, FL grapples with inherent heterogeneity challenges leading to issues such as stragglers, dropouts, and performance variations. Selection of clients to run an FL instance is crucial, but existing strategies introduce biases and participation issues and do not consider resource efficiency. Communication and training acceleration solutions proposed to increase client participation also fall short due to the dynamic nature of system resources. We address these challenges in this paper by designing FLOAT, a novel framework designed to boost FL client resource awareness. FLOAT optimizes resource utilization dynamically for meeting training deadlines, and mitigates stragglers and dropouts through various optimization techniques; leading to enhanced model convergence and improved performance. FLOAT leverages multi-objective Reinforcement Learning with Human Feedback (RLHF) to automate the selection of the optimization techniques and their configurations, tailoring them to individual client resource conditions. Moreover, FLOAT seamlessly integrates into existing FL systems, maintaining non-intrusiveness and versatility for both asynchronous and synchronous FL settings. As per our evaluations, FLOAT increases accuracy by up to 53%, reduces client dropouts by up to 78×, and improves communication, computation, and memory utilization by up to 81×, 44×, and 20× respectively.