CAREER: Federated Learning: Statistical Optimality and Provable Security

该奖项的全部或部分资金来自《2021年美国救援计划法案》(公法117-2)。机器学习和数据科学的快速发展迫使组织和个人越来越依赖数据来解决推理和决策问题。为了缓解数据所有者对隐私的担忧，研究人员和从业者一直在倡导一种新的学习范式--联合学习。在该框架下，中央学习器通过与分布式用户通信并将训练数据保存在本地用户处来训练模型。在为在不损害数据隐私的情况下训练机器学习模型开辟了新的机会的同时，由于分布式用户的异构性和不可靠性，联邦学习在维护统计效率和安全性方面面临着巨大的挑战。该项目的成功完成为高效和安全的联合学习提供了关键的使能技术，并加速了其在自动驾驶汽车和个性化医疗等安全和安全关键系统中的采用。拟议的教育活动包括专门针对代表性不足群体成员的研究生和本科生的教学和辅导，以及旨在提高公众隐私和安全意识的社区外联活动。这项研究开发了一个跨学科的项目，以调查联合学习的基本和算法方面，从统计效率到安全和隐私。分析了目前广泛采用的算法的统计效率，并分析了它们在到达固定点时的失败。基于元学习和聚类的新算法被开发出来，即使在存在模型和数据异构性的情况下，也被证明是统计上最优的。此外，本研究还对拜占庭攻击下的分散学习进行了全面的研究。借鉴稳健统计的思想，设计了指数收敛于最优误码率的拜占庭弹性梯度下降算法。最后，为了保护学习者的隐私免受窃听攻击，研究人员从信息论和学习者与对手的对偶性理论出发，通过创新的思想来设计最优的私人学习策略。作为理论研究的补充，新的学习算法以计算包的形式提供给联邦学习系统和真实数据应用程序。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Rapid developments in machine learning and data science have compelled organizations and individuals to rely more and more on data to solve inference and decision problems. To ease the privacy concerns of data owners, researchers and practitioners have been advocating a new learning paradigm – federated learning. Under this framework, the central learner trains a model by communicating with distributed users and keeping the training data stored locally at the users. While opening up a world of new opportunities for training machine-learning models without compromising data privacy, federated learning faces significant challenges in maintaining statistical efficiency and security due to the heterogeneity and unreliability of the distributed users. Successful completion of the project provides key enabling technologies for efficient and secure federated learning and accelerates its adoption in security- and safety-critical systems such as self-driving cars and personalized medicine. The proposed education activities include teaching and mentoring graduate and undergraduate students targeted specifically at members of under-represented groups and community outreach aiming at raising public privacy and security awareness. This research develops an interdisciplinary program to investigate the fundamental and algorithmic aspects of federated learning ranging from statistical efficiency to security and privacy. The statistical efficiency of the widely-adopted algorithms is analyzed beyond their failures of reaching stationary points. New algorithms are developed based on meta-learning and clustering, and shown to be statistically optimal even in the presence of model and data heterogeneity. Moreover, this research conducts a comprehensive study of decentralized learning under Byzantine attacks. By borrowing insights from robust statistics, byzantine-resilient gradient descent algorithms with exponential convergence to the optimal error rates are devised. Finally, to protect the learner's privacy against eavesdropping attacks, the investigator aims to design optimal private learning strategies by innovating ideas from information theory and duality theory between the learner and adversary. Complementing the theoretical investigation, the new learning algorithms are made available as computational packages for the federated learning systems and real-data applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.