CAREER: Robust Adaptive Optimization Algorithms for Differentially Private Learning

隐私保护优化算法是解决机器学习（ML）问题的重要工具，同时保护用于训练ML模型的数据集中的个人隐私。尽管最近取得了进展，但缺乏理解其性能的理论基础，因此由于效用问题，它们在实践中的使用受到限制。该项目旨在发展一种理论来理解私有优化器的性能，并使用它来指导具有可靠和稳健性能的算法设计。为此，该项目侧重于与差异化私人学习相关的三个主要挑战：(i)通过开发统一的理论框架来弥合理论与实践之间的差距，该框架可用于更好地理解和解释私人优化器的性能；（ii）应用该理论指导私有优化器的设计，其私密性和效用保证对超参数选择具有鲁棒性；（iii）将框架、既定原则和算法扩展到深度学习模型。该项目的新颖之处在于提供了一个统一的理论框架，可以对私有优化器进行严格的性能分析。通过提供理论基础，该项目将有助于加速对差异私人学习的研究，例如，允许有原则地设计鲁棒和可靠的训练算法。更广泛地说，这个项目有很大的潜力，通过提供工具来共享和分析敏感数据，而不必牺牲个人隐私，从而加速其他科学领域的进步。这个项目涉及研究生和本科生。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Privacy-preserving optimization algorithms are essential tools for solving machine learning (ML) problems while protecting the privacy of individuals in the datasets used for training ML models. Despite the recent advances, there is a lack of a theoretical foundation for understanding their performance and hence their use in practice is limited because of utility concerns. This project seeks to develop a theory to understand the performance of private optimizers and use it to guide the design of algorithms with reliable and robust performance. To this end, the project focuses on the three main challenges related to differentially private learning: (i) bridging the gap between theory and practice by developing a unified theoretical framework that can be used to better understand and explain the performance of private optimizers; (ii) applying the theory to guide the design of private optimizers whose privacy and utility guarantees have robustness to hyperparameter choices; (iii) extending the framework, established principles, and algorithms to deep learning models. The project’s novelty is in providing a unified theoretical framework that enables rigorous performance analysis of private optimizers. By providing a theoretical foundation, this project will help accelerate research on differentially private learning, for example, by allowing the principled design of robust and reliable training algorithms. More broadly, this project has a great potential to accelerate advances in other domains of science by providing tools to share and analyze sensitive data without having to sacrifice the privacy of individuals. The project involves both graduate and undergraduate students in this research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.