CCSS: Collaborative Research: Quality-Aware Distributed Computation for Wireless Federated Learning: Channel-Aware User Selection, Mini-Batch Size Adaptation, and Scheduling

点击翻译按钮获取中文摘要

With the explosive growth of ML/AI technologies, there is enormous potential to advance networking technologies to enable distributed ML/AI data analytics over networked systems. This project will explore innovative cross-disciplinary research at the intersections of wireless networking and machine learning, and study wireless federated learning (FL) for achieving collaborative intelligence in wireless networks. It will advance the fundamental understanding of quality-aware dynamic distributed computation and computation-communication co-design for wireless FL. This project will spur a new line of thinking and provide new insights to support various emerging ML/AI applications over wireless networked systems, such as collaborative robotics, multi-user mixed reality, and intelligent control and management of wireless networks. The proposed research will also be integrated with education activities at the PIs' institutions for graduate, undergraduate, and K-12 students via curriculum development, research experiences, and outreach. The PIs will make conscientious effort to recruit minority graduate students.This project will study quality-aware distributed computation for wireless FL, with focuses on channel-aware user selection, communication scheduling, and adaptive mini-batch size design. The proposed research is built on the key observation that the learning accuracy of the trained model in FL depends heavily on dynamic selection of users participating in the learning process and the quality of their local model updates (which is determined by their mini-batch sizes). The quality of local updates can be treated as a design parameter and used as a knob for adaptive control across users and over time based on users' communication and computation costs as well as capabilities. With this insight, the PIs will 1) quantify the impacts of the variances of users' local stochastic gradient updates on learning accuracy over the learning process, for general settings including non-IID data, non-convex loss functions, and asynchronous distributed learning; 2) develop adaptive algorithms that select the participating users and set their mini-batch sizes in each round of the FL algorithm, based on users' channel conditions and the impacts of their local updates on the training loss; 3) jointly design users' mini-batch sizes and schedule their communications to reduce the learning time, by investigating the intricate coupling between computation workloads and communication scheduling. Multi-objective optimization will be used to strike the right balance between learning accuracy and learning cost (or learning time).This project is jointly funded by the Division of Electrical, Communications and Cyber Systems (ECCS), and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.