CAREER: Federated Learning: Statistical Optimality and Provable Security

该奖项全部或部分由《2021年美国救援计划法案》（公法117-2）资助。机器学习和数据科学的快速发展迫使组织和个人越来越多地依赖数据来解决推理和决策问题。为了减轻数据所有者对隐私的担忧，研究人员和实践者一直在倡导一种新的学习范式——联邦学习。在该框架下，中心学习者通过与分布式用户通信来训练模型，并将训练数据存储在用户本地。虽然在不损害数据隐私的情况下为训练机器学习模型开辟了新的机会，但由于分布式用户的异质性和不可靠性，联邦学习在保持统计效率和安全性方面面临着重大挑战。该项目的成功完成为高效、安全的联邦学习提供了关键支持技术，并加速了其在安全关键系统（如自动驾驶汽车和个性化医疗）中的应用。拟议的教育活动包括专门针对代表性不足群体成员的研究生和本科生的教学和指导，以及旨在提高公众隐私和安全意识的社区外展活动。本研究开发了一个跨学科的项目来研究联邦学习的基础和算法方面，从统计效率到安全和隐私。分析了被广泛采用的算法在到达平稳点失败之外的统计效率。基于元学习和聚类的新算法被开发出来，即使在存在模型和数据异质性的情况下也显示出统计上的最优。此外，本研究对拜占庭攻击下的去中心化学习进行了全面的研究。通过借鉴鲁棒统计的见解，设计了具有指数收敛到最优错误率的拜占庭弹性梯度下降算法。最后，为了保护学习者的隐私免受窃听攻击，研究者旨在通过创新学习者和对手之间的信息论和对偶理论的思想来设计最佳的私人学习策略。作为理论研究的补充，新的学习算法作为计算包可用于联邦学习系统和实际数据应用。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Rapid developments in machine learning and data science have compelled organizations and individuals to rely more and more on data to solve inference and decision problems. To ease the privacy concerns of data owners, researchers and practitioners have been advocating a new learning paradigm – federated learning. Under this framework, the central learner trains a model by communicating with distributed users and keeping the training data stored locally at the users. While opening up a world of new opportunities for training machine-learning models without compromising data privacy, federated learning faces significant challenges in maintaining statistical efficiency and security due to the heterogeneity and unreliability of the distributed users. Successful completion of the project provides key enabling technologies for efficient and secure federated learning and accelerates its adoption in security- and safety-critical systems such as self-driving cars and personalized medicine. The proposed education activities include teaching and mentoring graduate and undergraduate students targeted specifically at members of under-represented groups and community outreach aiming at raising public privacy and security awareness. This research develops an interdisciplinary program to investigate the fundamental and algorithmic aspects of federated learning ranging from statistical efficiency to security and privacy. The statistical efficiency of the widely-adopted algorithms is analyzed beyond their failures of reaching stationary points. New algorithms are developed based on meta-learning and clustering, and shown to be statistically optimal even in the presence of model and data heterogeneity. Moreover, this research conducts a comprehensive study of decentralized learning under Byzantine attacks. By borrowing insights from robust statistics, byzantine-resilient gradient descent algorithms with exponential convergence to the optimal error rates are devised. Finally, to protect the learner's privacy against eavesdropping attacks, the investigator aims to design optimal private learning strategies by innovating ideas from information theory and duality theory between the learner and adversary. Complementing the theoretical investigation, the new learning algorithms are made available as computational packages for the federated learning systems and real-data applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.