Scalable Federated Learning and Analytics with Communication Efficiency in Mobile Cloud Computing

如今，移动的云计算成为常态，机器学习（ML）成为必需品。从智能手机到物联网（IoT）传感器，网络边缘的数百万台设备需要在云的帮助下自主协作以执行复杂的任务。这些复杂的任务，例如对象检测和问题回答，可能涉及边缘设备上的ML训练和分析。这项研究的重点是设计可扩展的，个性化的和通信高效的机制，用于下一代联邦学习和分析，并使用大量边缘设备。尽管联邦学习和联邦分析最近引起了广泛的关注，但它们仍处于研究的初级阶段，特别是随着边缘设备数量在移动的云计算系统中的扩展。大多数现有的研究都集中在传统的客户端-服务器架构，这是远远从可扩展性。随着设备数量的增加和网络流量负载的增加，设备在其能力和数据分布方面变得更加异构，并且通信效率更难实现。进一步加剧了这一挑战，因为共享的全局模型被训练，然后被部署用于推理，现有的工作没有考虑到个性化的需求，在个别设备上，定制其本地收集的数据集，并根据自己的要求。该研究计划的长期目标是通过大幅提高通信效率和训练个性化模型并将其用于本地分析的有效性来提高联邦学习和分析的可扩展性。为了实现这一长期目标，本研究提出了一些具体目标。首先，我们的目标是实现模型性能，隐私保护和通信效率之间的平衡权衡，并设计新的局部差分隐私，模型量化和聚合机制。其次，我们将研究新算法的设计，用于微调元学习模型，以满足每个边缘设备上的个性化需求。第三，我们将研究联合分析中大型复杂模型的推理工作负载如何最好地分布在多个边缘设备上，以满足其功率和能力限制。最后，为了证明这项研究的现实可行性，我们将构建一个生产质量的开源框架，其中包含我们在这项研究计划中提出的解决方案，以便它们可以在现实世界的生产环境中进行测试。该计划将在深度学习理论和分布式计算系统的交叉领域培训高素质人员（HQP），提供当今需求量很大的理论和实践技能组合。在该计划期间，将一贯强调公平，多样性和包容性（EDI）。

Today, mobile cloud computing becomes the norm and machine learning (ML) becomes a necessity. Millions of devices at the network edge, from smartphones to Internet of Things (IoT) sensors, need to collaborate autonomously to perform complex tasks with the assistance of the cloud. These complex tasks, such as object detection and question-answering, may involve both ML training and analytics at the edge devices. This research focuses on the design of scalable, personalized, and communication-efficient mechanisms for next-generation federated learning and analytics with large numbers of edge devices. Although both federated learning and federated analytics have drawn much recent attention, they remain at a nascent stage of research, especially as the number of edge devices scales up in mobile cloud computing systems. The majority of existing studies have focused on traditional client-server architectures, which is far from scalable. As the number of devices scales up and the network traffic load increases, devices become more heterogeneous in their capabilities and data distribution, and communication efficiency is far more difficult to achieve. Further exacerbating this challenge, as a shared global model is trained and then deployed for inference, existing work failed to consider the need for personalization on individual devices, customized for its locally collected dataset and tailored to its own requirements. The long-term goal of this research program is to improve the scalability of both federated learning and analytics, by substantially improving both communication efficiency and the effectiveness of training personalized models and using them for local analytics. Towards such a long-term goal, this research proposes to achieve a number of specific objectives. First, we aim to achieve a balanced trade-off between model performance, privacy preservation, and communication efficiency, and design new local differential privacy, model quantization and aggregation mechanisms. Second, we will investigate the design of new algorithms for fine-tuning meta-learning models for personalized for individual needs on each edge device. Third, we will study how inference workload with large and complex models in federated analytics can be best distributed across multiple edge devices to satisfy their power and capability constraints. Finally, to demonstrate real-world feasibility of this proposed research, we will build a production-quality open-source framework that encompasses our proposed solutions in this research program, so that they can be tested in a real-world production environment. The program will train highly qualified personnel (HQP) in the intersection of deep learning theory and distributed computing systems, offering a mix of theoretical and hands-on skill sets that are in high demand today. A strong emphasis on equity, diversity, and inclusion (EDI) will be consistently carried out during the program.