CAREER: Towards Efficient and Fast Hierarchical Federated Learning in Heterogeneous Wireless Edge Networks

职业：在异构无线边缘网络中实现高效快速的分层联邦学习

• 批准号: 2145031
• 负责人: Xiaowen Gong
• 金额: $ 50万
• 依托单位: Auburn University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145031&HistoricalAwards=false
• 关键词: CAREER; Towards; Efficient; Fast; Hierarchical

The accelerating penetration of machine learning (ML) based artificial intelligence (AI) in a variety of domains and the explosive growth of wireless applications spur wireless federated learning (WFL), which can achieve collaborative intelligence via federated learning (FL) in wireless edge networks. This project explores wireless hierarchical federated learning (WHFL), which leverages a hierarchical communication structure to substantially reduce the communication costs of WFL. It develops fundamental understandings as well as adaptive and efficient algorithms and schemes for WHFL while addressing several unique challenges that have been predominantly unexplored before: 1) inflexibility of homogeneous computation configurations (including local iteration number, mini-batch size, step size) for participating devices; 2) inefficiency of bandwidth-sharing based communication resource allocation; 3) complex impacts of the hierarchical communication structure for heterogeneous devices. The project explores innovative cross-disciplinary research of wireless networking and machine learning, and aims to provide useful insights of networking research for future intelligent networked computational systems based on data analytics. The research outcomes of this project have the potential to enable intelligent control and management of wireless networks, and also support various emerging AI applications over wireless networked systems, such as connected and autonomous vehicles, and collaborative robots. Various substantial education programs are integrated with the proposed research, including hands-on wireless and ML/AI projects for college students, and outreach activities on robotics for K-12 students.This project studies hierarchical FL in wireless edge networks for devices with heterogeneous computation and communication capabilities. The proposed research is motivated by some key insights obtained from our preliminary work: 1) heterogeneous computation configurations, particularly heterogeneous local iteration numbers, have non-trivial impacts on the learning accuracy and learning cost of FL; 2) time-sharing based communication resource allocation is more efficient than bandwidth-sharing, while it results in non-trivial coupling between computation configuration and communication scheduling; 3) the hierarchical model communication and aggregation have non-trivial impacts when devices have heterogeneous computation configurations. With these insights, the proposed research is organized into the following three interdependent thrusts: i) adaptive cost-aware device selection and computation configuration in a local cluster; ii) co-design of computation configuration and communication scheduling for fast convergence in a local cluster; iii) global model communication and aggregation for accurate learning across local clusters. The proposed schemes and algorithms for WHFL will be implemented to evaluate their practical performance. This project is jointly funded by CNS and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

基于机器学习（ML）的人工智能（AI）在各个领域的加速渗透以及无线应用的爆炸式增长刺激了无线联邦学习（WFL），它可以通过无线边缘网络中的联邦学习（FL）实现协作智能。本项目探索无线分层联邦学习（WHFL），它利用分层通信结构大幅降低WFL的通信成本。它为WHFL开发了基本的理解以及自适应和高效的算法和方案，同时解决了以前主要未探索的几个独特挑战：1)参与设备的同构计算配置（包括局部迭代次数，小批量大小，步长）的不灵活性；2)基于带宽共享的通信资源分配效率低下；3)层次通信结构对异构设备的复杂影响。该项目探索无线网络和机器学习的创新跨学科研究，旨在为基于数据分析的未来智能网络计算系统的网络研究提供有用的见解。该项目的研究成果有可能实现无线网络的智能控制和管理，并支持无线网络系统上的各种新兴人工智能应用，例如联网和自动驾驶汽车以及协作机器人。各种实质性的教育项目与拟议的研究相结合，包括面向大学生的动手无线和ML/AI项目，以及面向K-12学生的机器人拓展活动。本项目研究具有异构计算和通信能力的设备在无线边缘网络中的分层FL。本文提出的研究是基于我们从前期工作中获得的一些关键见解：1)异构计算配置，特别是异构局部迭代数，对FL的学习精度和学习成本有不小的影响；2)基于分时的通信资源分配比基于带宽共享的通信资源分配效率更高，但会导致计算配置与通信调度之间的非平凡耦合；3)当设备具有异构计算配置时，层次模型的通信和聚合具有重要的影响。有了这些见解，提出的研究分为以下三个相互依存的重点：i)自适应成本感知设备选择和本地集群的计算配置；Ii)协同设计计算配置和通信调度，实现局部集群的快速收敛；Iii)全局模型通信和聚合，以实现跨局部集群的准确学习。本文将对所提出的WHFL方案和算法进行实施，以评估其实际性能。该项目由CNS和促进竞争研究的既定计划（EPSCoR）共同资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Anarchic Convex Federated Learning

• DOI: 10.1109/infocomwkshps57453.2023.10225908
• 发表时间: 2023-05
• 期刊: IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)
• 作者: Dongsheng Li;Xiaowen Gong

Anarchic Federated learning with Delayed Gradient Averaging

• DOI: 10.1145/3565287.3610273
• 发表时间: 2023-10
• 期刊: Proceedings of the Twenty-fourth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing
• 作者: Dongsheng Li;Xiaowen Gong