Collaborative Research: SaTC: CORE: Small: Critical Learning Periods Augmented Robust Federated Learning

联合学习(FL)是一种分布式机器学习方法，它允许多个数据所有者(客户)协作训练机器学习模型，这些模型受益于每个所有者的数据，而不必共享数据本身。联合学习可以改善隐私并保护受限数据，这使其在医疗保健、金融科技和自动驾驶等领域成为一种有吸引力的工具。然而，联合学习受到关键学习(CL)阶段的影响：最初几轮培训对模型的质量和稳健性有过大的影响。如果能够开发出用于选择和加权来自不同客户的贡献的新方法来解决CL周期的原因，则CL周期可能有助于联合学习系统提高模型质量。然而，它们也为攻击者提供了机会，他们可能能够利用CL时间段来发动更精确和更有影响力的攻击。为了更好地了解这些机会和攻击，本项目将对CL周期的特征和可利用性进行全面分析，以推进联邦学习的健壮性和脆弱性的研究。该团队将开发数据集、模型、算法和系统源代码，并与研究社区共享，同时科学成果将以研究论文、技术报告、书籍章节、课程材料和教程的形式广泛传播。本科生，特别是来自代表性不足群体的本科生，将参与拟议的研究活动。该项目的中心目标是调查和了解FL训练过程中的CL周期，利用CL周期的独特属性来增强FL的安全性和健壮性，同时发现攻击者可以利用的漏洞。为了实现这一目标，该项目调查了三个主要主题。第一个主题集中在如何在外语训练中及时有效地识别CL周期和相关漏洞。第二个主题集中在如何利用CL周期意识来优化FL模型的精度，重点关注自适应客户选择的方法，这些方法是针对第一个主题中发展的CL周期的原因而调整的。第三个主题研究了如何将主题1中的研究结果推广到其他流行的外语技术，如梯度压缩、公平聚合、个性化及其联合效应，以解决系统异质性(例如，通信带宽差异、异质本地模型和公平性问题)。同时，该团队还将设计和开发一个强大的FL试验台，以使用真实世界的模型和数据集对建议的算法进行经验性评估。该项目由安全可信的网络空间和既定的激励竞争研究计划(EPSCoR)联合资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Federated Learning (FL) is a distributed machine learning approach that allows multiple data owners ("clients") to collaboratively train machine learning models that benefit from each owner's data without having to share the data itself. Federated learning can improve privacy and protect restricted data, which makes it an attractive tool in sectors such as healthcare, fintech, and autonomous driving. However, federated learning is subject to critical learning (CL) periods: the initial rounds of training have an outsized impact on models' quality and robustness. CL periods may help federated learning systems improve model quality, if new methods for selecting and weighting contributions from different clients can be developed to address the causes of CL periods. However, they also present opportunities for attackers, who may be able to harness CL periods to launch more precise and impactful attacks. To better understand these opportunities and attacks, this project will conduct a comprehensive analysis of the characteristics and exploitability of CL periods so as to advance the study of the robustness and vulnerability of federated learning. The team will develop datasets, models, algorithms, and system source code and share it with the research community, while the scientific findings will be widely disseminated as research papers, technical reports, book chapters, course materials, and tutorials. Undergraduate students, particularly those from under-represented groups, will be engaged in the proposed research activities. The central goal of this project is to investigate and understand CL periods during the FL training process, exploiting unique properties of CL periods to enhance FL security and robustness while uncovering vulnerabilities that attackers could exploit. To achieve this objective, the project investigates three main themes. The first theme focuses on how to efficiently identify CL periods and related vulnerabilities in a timely manner during FL training. The second theme focuses on how to optimize FL model accuracy with CL periods awareness, focusing on methods for adaptive client selection that are tuned to the causes of CL periods developed in the first theme. The third theme investigates ways to generalize the findings from Theme 1 to other popular FL techniques such as gradient compression, fair aggregation, personalization, and their joint effect, to address system heterogeneity (e.g., communication bandwidth differences, heterogeneous local models, and fairness concerns). Concurrently with the three main themes, the team will also design and develop a robust FL testbed to empirically evaluate the proposed algorithms with real-world models and datasets.This project is jointly funded by Secure and Trustworthy Cyberspace and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.