CCSS: Collaborative Research: Quality-Aware Distributed Computation for Wireless Federated Learning: Channel-Aware User Selection, Mini-Batch Size Adaptation, and Scheduling

CCSS：协作研究：无线联邦学习的质量感知分布式计算：通道感知用户选择、小批量大小自适应和调度

• 批准号: 2121215
• 负责人: Xiaowen Gong
• 金额: $ 22万
• 依托单位: Auburn University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2121215&HistoricalAwards=false
• 关键词: CCSS; Collaborative; Research; Quality; Aware

With the explosive growth of ML/AI technologies, there is enormous potential to advance networking technologies to enable distributed ML/AI data analytics over networked systems. This project will explore innovative cross-disciplinary research at the intersections of wireless networking and machine learning, and study wireless federated learning (FL) for achieving collaborative intelligence in wireless networks. It will advance the fundamental understanding of quality-aware dynamic distributed computation and computation-communication co-design for wireless FL. This project will spur a new line of thinking and provide new insights to support various emerging ML/AI applications over wireless networked systems, such as collaborative robotics, multi-user mixed reality, and intelligent control and management of wireless networks. The proposed research will also be integrated with education activities at the PIs' institutions for graduate, undergraduate, and K-12 students via curriculum development, research experiences, and outreach. The PIs will make conscientious effort to recruit minority graduate students.This project will study quality-aware distributed computation for wireless FL, with focuses on channel-aware user selection, communication scheduling, and adaptive mini-batch size design. The proposed research is built on the key observation that the learning accuracy of the trained model in FL depends heavily on dynamic selection of users participating in the learning process and the quality of their local model updates (which is determined by their mini-batch sizes). The quality of local updates can be treated as a design parameter and used as a knob for adaptive control across users and over time based on users' communication and computation costs as well as capabilities. With this insight, the PIs will 1) quantify the impacts of the variances of users' local stochastic gradient updates on learning accuracy over the learning process, for general settings including non-IID data, non-convex loss functions, and asynchronous distributed learning; 2) develop adaptive algorithms that select the participating users and set their mini-batch sizes in each round of the FL algorithm, based on users' channel conditions and the impacts of their local updates on the training loss; 3) jointly design users' mini-batch sizes and schedule their communications to reduce the learning time, by investigating the intricate coupling between computation workloads and communication scheduling. Multi-objective optimization will be used to strike the right balance between learning accuracy and learning cost (or learning time).This project is jointly funded by the Division of Electrical, Communications and Cyber Systems (ECCS), and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着ML/AI技术的爆炸式增长，推进网络技术以实现网络系统上的分布式ML/AI数据分析具有巨大的潜力。该项目将在无线网络和机器学习的交叉点探索创新的跨学科研究，并研究无线联邦学习（FL），以实现无线网络中的协作智能。它将推进对无线FL的质量感知动态分布式计算和计算通信协同设计的基本理解。该项目将激发新的思维，并提供新的见解，以支持无线网络系统上各种新兴的ML/AI应用，如协作机器人，多用户混合现实以及无线网络的智能控制和管理。拟议的研究还将通过课程开发，研究经验和推广活动，与PI机构的研究生，本科生和K-12学生的教育活动相结合。本项目将研究无线FL的质量感知分布式计算，重点研究信道感知用户选择、通信调度和自适应小批量设计。拟议的研究是建立在一个关键的观察基础上的，即FL中训练模型的学习精度在很大程度上取决于参与学习过程的用户的动态选择及其本地模型更新的质量（由其小批量大小决定）。本地更新的质量可以被视为设计参数，并用作基于用户的通信和计算成本以及能力的跨用户和随时间的自适应控制的旋钮。有了这种洞察力，PI将1）量化用户的局部随机梯度更新的方差对学习过程中的学习精度的影响，对于包括非IID数据、非凸损失函数和异步分布式学习的一般设置; 2）开发自适应算法，该自适应算法在FL算法的每一轮中选择参与用户并设置其小批量大小，基于用户的信道条件和他们的本地更新对训练损失的影响; 3）通过研究计算工作量和通信调度之间的复杂耦合，联合设计用户的小批量大小和调度他们的通信以减少学习时间。多目标优化将用于在学习精度和学习成本之间取得适当的平衡（或学习时间）。本项目由电气、通信和网络系统司（ECCS）联合资助，激励竞争研究计划（EPSCoR）该奖项反映了NSF的法定使命，并通过使用基金会的智力价值进行评估，更广泛的影响审查标准。

项目成果

Quality-Aware Distributed Computation for Cost-Effective Non-Convex and Asynchronous Wireless Federated Learning

• DOI: 10.23919/wiopt52861.2021.9589660
• 发表时间: 2021-10
• 期刊: 2021 19th International Symposium on Modeling and Optimization in Mobile, Ad hoc, and Wireless Networks (WiOpt)
• 作者: Yuxi Zhao;Xiaowen Gong

Quality-Aware Distributed Computation and Communication Scheduling for Fast Convergent Wireless Federated Learning

• DOI: 10.23919/wiopt52861.2021.9589802
• 发表时间: 2021-10
• 期刊: 2021 19th International Symposium on Modeling and Optimization in Mobile, Ad hoc, and Wireless Networks (WiOpt)
• 作者: Dongsheng Li;Yuxi Zhao;Xiaowen Gong

Truthful Incentive Mechanism for Federated Learning with Crowdsourced Data Labeling

• DOI: 10.1109/infocom53939.2023.10228923
• 发表时间: 2023-01
• 期刊: IEEE INFOCOM 2023 - IEEE Conference on Computer Communications
• 作者: Yuxi Zhao;Xiaowen Gong;S. Mao

Distributed Policy Gradient with Heterogeneous Computations for Federated Reinforcement Learning

• DOI: 10.1109/ciss56502.2023.10089771
• 发表时间: 2023-03
• 期刊: 2023 57th Annual Conference on Information Sciences and Systems (CISS)
• 作者: Ye Zhu;Xiaowen Gong