CAREER: Robust Adaptive Optimization Algorithms for Differentially Private Learning

职业：用于差异化私人学习的鲁棒自适应优化算法

• 批准号: 1943046
• 负责人: Jaewoo Lee
• 金额: $ 52.96万
• 依托单位: University of Georgia Research Foundation Inc
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-03-01 至 2025-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1943046&HistoricalAwards=false
• 关键词: CAREER; Robust; Adaptive; Optimization; Algorithms

Privacy-preserving optimization algorithms are essential tools for solving machine learning (ML) problems while protecting the privacy of individuals in the datasets used for training ML models. Despite the recent advances, there is a lack of a theoretical foundation for understanding their performance and hence their use in practice is limited because of utility concerns. This project seeks to develop a theory to understand the performance of private optimizers and use it to guide the design of algorithms with reliable and robust performance. To this end, the project focuses on the three main challenges related to differentially private learning: (i) bridging the gap between theory and practice by developing a unified theoretical framework that can be used to better understand and explain the performance of private optimizers; (ii) applying the theory to guide the design of private optimizers whose privacy and utility guarantees have robustness to hyperparameter choices; (iii) extending the framework, established principles, and algorithms to deep learning models. The project’s novelty is in providing a unified theoretical framework that enables rigorous performance analysis of private optimizers. By providing a theoretical foundation, this project will help accelerate research on differentially private learning, for example, by allowing the principled design of robust and reliable training algorithms. More broadly, this project has a great potential to accelerate advances in other domains of science by providing tools to share and analyze sensitive data without having to sacrifice the privacy of individuals. The project involves both graduate and undergraduate students in this research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

隐私保护优化算法是解决机器学习（ML）问题的重要工具，同时保护用于训练ML模型的数据集中的个人隐私。尽管最近取得了进展，但缺乏理解其性能的理论基础，因此由于实用性问题，它们在实践中的使用受到限制。该项目旨在开发一种理论来理解私有优化器的性能，并使用它来指导具有可靠和鲁棒性能的算法的设计。为此，该项目侧重于与差异化私人学习相关的三个主要挑战：（一）通过开发一个统一的理论框架来弥合理论与实践之间的差距，该框架可用于更好地理解和解释私人优化器的性能;（二）应用该理论指导私人优化器的设计，其隐私和效用保证对超参数选择具有鲁棒性;（iii）将框架、既定原则和算法扩展到深度学习模型。该项目的新奇之处在于提供了一个统一的理论框架，可以对私有优化器进行严格的性能分析。通过提供理论基础，该项目将有助于加速对差异化私人学习的研究，例如，通过允许稳健和可靠的训练算法的原则设计。更广泛地说，该项目有很大的潜力，通过提供工具来共享和分析敏感数据，而不必牺牲个人隐私，从而加速其他科学领域的进步。这个奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Stochastic Adaptive Line Search for Differentially Private Optimization

• DOI: 10.1109/bigdata50022.2020.9378011
• 发表时间: 2020-08
• 期刊: 2020 IEEE International Conference on Big Data (Big Data)
• 作者: Chen Chen-Chen;Jaewoo Lee

Scaling up Differentially Private Deep Learning with Fast Per-Example Gradient Clipping

• DOI: 10.2478/popets-2021-0008
• 发表时间: 2020-09
• 期刊: Proceedings on Privacy Enhancing Technologies
• 作者: Jaewoo Lee;Daniel Kifer

Differentially Private Goodness-of-Fit Tests for Continuous Variables

• DOI: 10.1016/j.ecosta.2021.09.007
• 发表时间: 2021-10
• 期刊: Econometrics and Statistics
• 影响因子: 1.9
• 作者: Seungwoo Kwak;Jeongyoun Ahn;Jaewoo Lee;Cheolwoo Park

MBAG: A Scalable Mini-Block Adaptive Gradient Method for Deep Neural Networks

• DOI: 10.1109/bigdata55660.2022.10020262
• 发表时间: 2022-12
• 期刊: 2022 IEEE International Conference on Big Data (Big Data)
• 作者: Jaewoo Lee

Performance Testing for Cloud Computing with Dependent Data Bootstrapping

使用依赖数据引导的云计算性能测试

• DOI: 10.1109/ase51524.2021.9678687
• 发表时间: 2021
• 作者: He, Sen;Liu, Tianyi;Lama, Palden;Lee, Jaewoo;Kim, In Kee;Wang, Wei
• 通讯作者: Wang, Wei