EAGER: Distributed Heterogeneous Data Analytics via Federated Learning

EAGER：通过联邦学习进行分布式异构数据分析

• 批准号: 2140247
• 负责人: Yiran Chen
• 金额: $ 15万
• 依托单位: Duke University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2140247&HistoricalAwards=false
• 关键词: EAGER; Distributed; Heterogeneous; Data; Analytics

With the growth of devices in the Internet of Things (IoT), a huge amount of data are generated at the network edge. This provides valuable resources for learning insightful information and enabling intelligent applications such as, self-driving, video analytics, anomaly detection, etc. Federated learning (FL) is a promising technique that enables a large number of clients orchestrated by a central server to collaboratively learn a machine learning model without sharing data. However, the data owned by different devices are typically not independent and identically distributed (non-IID) due to different user preferences and usage patterns. Conventional FL methods fail to generalize well for most clients. In addition to data heterogeneity, system heterogeneity; that is, where clients have different computation and communication capabilities, is another critical challenge for FL development. Because the central server does not perform the aggregation until receiving all the clients’ updates, system heterogeneity significantly slows down the model training if the clients are randomly selected to participate in the training. The goal of this research is to develop a unified FL framework for addressing both data and system heterogeneity at the same time.This project will pave the foundations for properly handling data and system heterogeneity in FL with three integrated components: 1) unveiling essential reasons of performance degradation in FL with non-IID data; 2) exploring comprehensive principles to guide the client composition for FL with non-IID data; and 3) developing a unified FL method for addressing both data and system heterogeneity simultaneously, including a client utility function and a reinforcement learning based client composition method. This project will develop and train undergraduate and graduate researchers with comprehensive experience for developing FL systems, including recruiting minority and under-represented students. The outcome of this project will be incorporated in both new and existing undergraduate and graduate courses at Duke University.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着物联网（IoT）中设备的增长，在网络边缘生成了大量数据。这为学习有洞察力的信息和实现智能应用提供了宝贵的资源，例如自动驾驶，视频分析，异常检测等。联合学习（FL）是一种很有前途的技术，它使中央服务器协调的大量客户端能够在不共享数据的情况下协作学习机器学习模型。然而，由于不同的用户偏好和使用模式，不同设备所拥有的数据通常不是独立和相同分布的（非IID）。传统的FL方法不能很好地推广到大多数客户。除了数据异构性之外，系统异构性，即客户端具有不同的计算和通信能力，是FL开发的另一个关键挑战。 由于中央服务器在接收到所有客户端的更新之前不会执行聚合，因此如果随机选择客户端来参与训练，则系统异构性会显著减慢模型训练。本研究的目标是开发一个统一的FL框架，以同时解决数据和系统异构性，该项目将为正确处理FL中的数据和系统异构性奠定基础，包括三个方面：1）揭示非IID数据FL性能下降的根本原因; 2）探索指导非IID数据FL客户端组合的综合原则;以及3）开发一种同时解决数据和系统异构性的统一FL方法，包括客户端效用函数和基于强化学习的客户端组合方法。该项目将开发和培训具有开发外语系统综合经验的本科生和研究生研究人员，包括招募少数民族和代表性不足的学生。该项目的成果将被纳入杜克大学新的和现有的本科生和研究生课程中。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

项目成果

FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective

• 发表时间: 2021-10
• 作者: Jingwei Sun;Ang Li;Louis DiValentin;Amin Hassanzadeh;Yiran Chen;H. Li

Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction

• DOI: 10.48550/arxiv.2209.15245
• 发表时间: 2022-09
• 作者: Jianyi Zhang;Ang Li;Minxue Tang;Jingwei Sun;Xiang Chen;Fan Zhang;Chang Chen;Yiran Chen;H. Li

Mixture Outlier Exposure: Towards Out-of-Distribution Detection in Fine-grained Environments

• DOI: 10.1109/wacv56688.2023.00549
• 发表时间: 2021-06
• 期刊: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
• 作者: Jingyang Zhang;Nathan Inkawhich;Randolph Linderman;Yiran Chen;H. Li

ReAugKD: Retrieval-Augmented Knowledge Distillation For Pre-trained Language Models

ReAugKD：预训练语言模型的检索增强知识蒸馏

• 发表时间: 2023
• 期刊: The 61st Annual Meeting of the Association for Computational Linguistics
• 作者: Zhang, Jianyi;Muhamed, Aashiq;Anantharaman, Aditya;Wang, Guoyin;Chen, Changyou;Zhong, Kai;Cui, Qingjun;Xu, Yi;Zeng, Belinda;Chilimbi, Trishul
• 通讯作者: Chilimbi, Trishul

An Audio Frequency Unfolding Framework for Ultra-Low Sampling Rate Sensors

超低采样率传感器的音频展开框架

• DOI: 10.1109/isqed54688.2022.9806149
• 发表时间: 2022
• 期刊: 2022 23rd International Symposium on Quality Electronic Design
• 作者: Gao, Zhihui;Tang, Minxue;Li, Ang;Chen, Yiran
• 通讯作者: Chen, Yiran