CAREER: Towards Efficient and Fast Hierarchical Federated Learning in Heterogeneous Wireless Edge Networks

基于机器学习（ML）的人工智能（AI）在各个领域的加速渗透和无线应用的爆炸式增长刺激了无线联邦学习（WFL），其可以通过无线边缘网络中的联邦学习（FL）来实现协作智能。该项目探讨了无线分层联邦学习（WHFL），它利用分层通信结构来大幅降低WFL的通信成本。它为WHFL开发了基本的理解以及自适应和有效的算法和方案，同时解决了几个独特的挑战，这些挑战以前主要是未探索的：1）同构计算配置的可扩展性（包括本地迭代次数、小批量大小、步长）; 2）基于带宽共享的通信资源分配的低效率; 3）异构设备的分层通信结构的复杂影响。该项目探索无线网络和机器学习的创新性跨学科研究，旨在为未来基于数据分析的智能网络计算系统提供有用的网络研究见解。该项目的研究成果有可能实现无线网络的智能控制和管理，并支持无线网络系统上的各种新兴人工智能应用，如联网和自动驾驶汽车以及协作机器人。本研究结合了多个实质性的教育项目，包括面向大学生的无线和ML/AI实践项目，以及面向K-12学生的机器人拓展活动。本项目针对具有异构计算和通信能力的设备，研究无线边缘网络中的分层FL。本文的研究工作主要基于以下几点：1）异构计算配置，特别是异构局部迭代次数，对FL的学习精度和学习代价有重要影响; 2）基于分时的通信资源分配比带宽共享更有效，但计算配置和通信调度之间存在重要耦合; 3）当设备具有异构计算配置时，分层模型通信和聚合具有重要影响。有了这些见解，所提出的研究被组织成以下三个相互依存的推力：i）自适应成本感知的设备选择和计算配置在本地集群; ii）计算配置和通信调度的协同设计，在本地集群中快速收敛; iii）全局模型通信和聚合，跨本地集群的准确学习。所提出的计划和算法的WHFL将实施，以评估其实际性能。该项目由CNS和刺激竞争研究的既定计划（EPSCoR）共同资助。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The accelerating penetration of machine learning (ML) based artificial intelligence (AI) in a variety of domains and the explosive growth of wireless applications spur wireless federated learning (WFL), which can achieve collaborative intelligence via federated learning (FL) in wireless edge networks. This project explores wireless hierarchical federated learning (WHFL), which leverages a hierarchical communication structure to substantially reduce the communication costs of WFL. It develops fundamental understandings as well as adaptive and efficient algorithms and schemes for WHFL while addressing several unique challenges that have been predominantly unexplored before: 1) inflexibility of homogeneous computation configurations (including local iteration number, mini-batch size, step size) for participating devices; 2) inefficiency of bandwidth-sharing based communication resource allocation; 3) complex impacts of the hierarchical communication structure for heterogeneous devices. The project explores innovative cross-disciplinary research of wireless networking and machine learning, and aims to provide useful insights of networking research for future intelligent networked computational systems based on data analytics. The research outcomes of this project have the potential to enable intelligent control and management of wireless networks, and also support various emerging AI applications over wireless networked systems, such as connected and autonomous vehicles, and collaborative robots. Various substantial education programs are integrated with the proposed research, including hands-on wireless and ML/AI projects for college students, and outreach activities on robotics for K-12 students.This project studies hierarchical FL in wireless edge networks for devices with heterogeneous computation and communication capabilities. The proposed research is motivated by some key insights obtained from our preliminary work: 1) heterogeneous computation configurations, particularly heterogeneous local iteration numbers, have non-trivial impacts on the learning accuracy and learning cost of FL; 2) time-sharing based communication resource allocation is more efficient than bandwidth-sharing, while it results in non-trivial coupling between computation configuration and communication scheduling; 3) the hierarchical model communication and aggregation have non-trivial impacts when devices have heterogeneous computation configurations. With these insights, the proposed research is organized into the following three interdependent thrusts: i) adaptive cost-aware device selection and computation configuration in a local cluster; ii) co-design of computation configuration and communication scheduling for fast convergence in a local cluster; iii) global model communication and aggregation for accurate learning across local clusters. The proposed schemes and algorithms for WHFL will be implemented to evaluate their practical performance. This project is jointly funded by CNS and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.