Scalable Federated Learning and Analytics with Communication Efficiency in Mobile Cloud Computing

如今，移动云计算成为常态，机器学习（ML）成为一种必需品。数以百万计的网络边缘设备，从智能手机到物联网（IoT）传感器，需要在云的帮助下自主协作，执行复杂的任务。这些复杂的任务，如对象检测和问答，可能涉及边缘设备的机器学习训练和分析。本研究的重点是设计可扩展、个性化和通信高效的机制，用于下一代具有大量边缘设备的联邦学习和分析。尽管联邦学习和联邦分析最近都引起了很多关注，但它们仍处于研究的初级阶段，尤其是在移动云计算系统中边缘设备数量不断增加的情况下。现有的大多数研究都集中在传统的客户机-服务器体系结构上，这种体系结构远远不能扩展。随着设备数量的增加和网络流量负载的增加，设备的功能和数据分布变得更加异构，通信效率更难实现。进一步加剧这一挑战的是，当一个共享的全局模型被训练并部署用于推理时，现有的工作未能考虑到对单个设备的个性化需求，针对其本地收集的数据集进行定制，并根据自己的需求进行定制。该研究计划的长期目标是通过大幅提高通信效率和训练个性化模型的有效性，并将其用于本地分析，来提高联邦学习和分析的可扩展性。为了实现这样一个长期目标，本研究提出了一些具体的目标。首先，我们的目标是实现模型性能、隐私保护和通信效率之间的平衡，并设计新的局部差分隐私、模型量化和聚合机制。其次，我们将研究新算法的设计，以微调元学习模型，以满足每个边缘设备上的个性化需求。第三，我们将研究联邦分析中大型复杂模型的推理工作负载如何最好地分布在多个边缘设备上，以满足它们的功率和能力限制。最后，为了证明这项研究在现实世界中的可行性，我们将构建一个生产质量的开源框架，其中包含我们在该研究计划中提出的解决方案，以便它们可以在现实世界的生产环境中进行测试。该项目将培养深度学习理论和分布式计算系统交叉领域的高素质人才（HQP），提供当今高需求的理论和实践技能组合。在项目期间，将始终强调公平、多样性和包容性。

Today, mobile cloud computing becomes the norm and machine learning (ML) becomes a necessity. Millions of devices at the network edge, from smartphones to Internet of Things (IoT) sensors, need to collaborate autonomously to perform complex tasks with the assistance of the cloud. These complex tasks, such as object detection and question-answering, may involve both ML training and analytics at the edge devices. This research focuses on the design of scalable, personalized, and communication-efficient mechanisms for next-generation federated learning and analytics with large numbers of edge devices. Although both federated learning and federated analytics have drawn much recent attention, they remain at a nascent stage of research, especially as the number of edge devices scales up in mobile cloud computing systems. The majority of existing studies have focused on traditional client-server architectures, which is far from scalable. As the number of devices scales up and the network traffic load increases, devices become more heterogeneous in their capabilities and data distribution, and communication efficiency is far more difficult to achieve. Further exacerbating this challenge, as a shared global model is trained and then deployed for inference, existing work failed to consider the need for personalization on individual devices, customized for its locally collected dataset and tailored to its own requirements. The long-term goal of this research program is to improve the scalability of both federated learning and analytics, by substantially improving both communication efficiency and the effectiveness of training personalized models and using them for local analytics. Towards such a long-term goal, this research proposes to achieve a number of specific objectives. First, we aim to achieve a balanced trade-off between model performance, privacy preservation, and communication efficiency, and design new local differential privacy, model quantization and aggregation mechanisms. Second, we will investigate the design of new algorithms for fine-tuning meta-learning models for personalized for individual needs on each edge device. Third, we will study how inference workload with large and complex models in federated analytics can be best distributed across multiple edge devices to satisfy their power and capability constraints. Finally, to demonstrate real-world feasibility of this proposed research, we will build a production-quality open-source framework that encompasses our proposed solutions in this research program, so that they can be tested in a real-world production environment. The program will train highly qualified personnel (HQP) in the intersection of deep learning theory and distributed computing systems, offering a mix of theoretical and hands-on skill sets that are in high demand today. A strong emphasis on equity, diversity, and inclusion (EDI) will be consistently carried out during the program.