EAGER: Distributed Heterogeneous Data Analytics via Federated Learning

随着物联网(IoT)设备的增长，网络边缘产生了海量数据。这为学习有洞察力的信息和实现智能应用(如自动驾驶、视频分析、异常检测等)提供了宝贵的资源。联合学习(FL)是一种很有前途的技术，它使中央服务器协调的大量客户端能够在不共享数据的情况下协作学习机器学习模型。然而，由于不同的用户偏好和使用模式，不同设备拥有的数据通常不是独立和相同分布的(非IID)。传统的FL方法不能很好地推广到大多数客户。除了数据异构性之外，系统异构性也是FL开发面临的另一个关键挑战。由于中央服务器直到接收到所有客户端的更新才执行聚合，因此如果随机选择客户端参与训练，系统异构性会显著减慢模型训练的速度。本研究的目标是开发一个同时处理数据和系统异构性的统一FL框架。该项目将通过三个集成组件为正确处理FL中的数据和系统异构性奠定基础：1)揭示非IID数据FL性能下降的根本原因；2)探索指导包含非IID数据的FL客户端组合的综合原则；3)开发同时处理数据和系统异构性的统一FL方法，包括客户端效用函数和基于强化学习的客户端组合方法。该项目将培养和培训具有全面经验的本科生和研究生研究人员开发外语系统，包括招收少数族裔和代表性不足的学生。该项目的成果将被纳入杜克大学新的和现有的本科生和研究生课程。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

With the growth of devices in the Internet of Things (IoT), a huge amount of data are generated at the network edge. This provides valuable resources for learning insightful information and enabling intelligent applications such as, self-driving, video analytics, anomaly detection, etc. Federated learning (FL) is a promising technique that enables a large number of clients orchestrated by a central server to collaboratively learn a machine learning model without sharing data. However, the data owned by different devices are typically not independent and identically distributed (non-IID) due to different user preferences and usage patterns. Conventional FL methods fail to generalize well for most clients. In addition to data heterogeneity, system heterogeneity; that is, where clients have different computation and communication capabilities, is another critical challenge for FL development. Because the central server does not perform the aggregation until receiving all the clients’ updates, system heterogeneity significantly slows down the model training if the clients are randomly selected to participate in the training. The goal of this research is to develop a unified FL framework for addressing both data and system heterogeneity at the same time.This project will pave the foundations for properly handling data and system heterogeneity in FL with three integrated components: 1) unveiling essential reasons of performance degradation in FL with non-IID data; 2) exploring comprehensive principles to guide the client composition for FL with non-IID data; and 3) developing a unified FL method for addressing both data and system heterogeneity simultaneously, including a client utility function and a reinforcement learning based client composition method. This project will develop and train undergraduate and graduate researchers with comprehensive experience for developing FL systems, including recruiting minority and under-represented students. The outcome of this project will be incorporated in both new and existing undergraduate and graduate courses at Duke University.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.