Scalable Federated Learning and Analytics with Communication Efficiency in Mobile Cloud Computing

点击翻译按钮获取中文摘要

Today, mobile cloud computing becomes the norm and machine learning (ML) becomes a necessity. Millions of devices at the network edge, from smartphones to Internet of Things (IoT) sensors, need to collaborate autonomously to perform complex tasks with the assistance of the cloud. These complex tasks, such as object detection and question-answering, may involve both ML training and analytics at the edge devices. This research focuses on the design of scalable, personalized, and communication-efficient mechanisms for next-generation federated learning and analytics with large numbers of edge devices. Although both federated learning and federated analytics have drawn much recent attention, they remain at a nascent stage of research, especially as the number of edge devices scales up in mobile cloud computing systems. The majority of existing studies have focused on traditional client-server architectures, which is far from scalable. As the number of devices scales up and the network traffic load increases, devices become more heterogeneous in their capabilities and data distribution, and communication efficiency is far more difficult to achieve. Further exacerbating this challenge, as a shared global model is trained and then deployed for inference, existing work failed to consider the need for personalization on individual devices, customized for its locally collected dataset and tailored to its own requirements. The long-term goal of this research program is to improve the scalability of both federated learning and analytics, by substantially improving both communication efficiency and the effectiveness of training personalized models and using them for local analytics. Towards such a long-term goal, this research proposes to achieve a number of specific objectives. First, we aim to achieve a balanced trade-off between model performance, privacy preservation, and communication efficiency, and design new local differential privacy, model quantization and aggregation mechanisms. Second, we will investigate the design of new algorithms for fine-tuning meta-learning models for personalized for individual needs on each edge device. Third, we will study how inference workload with large and complex models in federated analytics can be best distributed across multiple edge devices to satisfy their power and capability constraints. Finally, to demonstrate real-world feasibility of this proposed research, we will build a production-quality open-source framework that encompasses our proposed solutions in this research program, so that they can be tested in a real-world production environment. The program will train highly qualified personnel (HQP) in the intersection of deep learning theory and distributed computing systems, offering a mix of theoretical and hands-on skill sets that are in high demand today. A strong emphasis on equity, diversity, and inclusion (EDI) will be consistently carried out during the program.