SaTC: CORE: Small: Auditing Private Statistical and Machine Learning Algorithms: Theory and Practice

使用统计学和机器学习来分析敏感数据会给那些将数据贡献给这些算法的用户带来严重的隐私风险，包括：数据重建（揭示大部分训练数据）、成员推断（揭示训练数据中特定个体的存在）和数据记忆（揭示特定训练示例）。差分隐私现在已经成为机器学习和统计领域保护数据隐私的标准，因为它为个人隐私提供了强有力的、正式的、定量的保证。然而，对于许多差异隐私的部署，在实践中形式保证和期望之间仍然存在显着差距，本项目旨在通过审计部署的算法来发现和解释其隐私属性，从而弥合这一差距。该项目的新颖之处在于建立了隐私审计的理论基础，并设计了测量现实场景中私有算法隐私泄漏的经验方法。该项目更广泛的意义和重要性将在于向从业者提供关于选择和使用私有算法的具体建议，以及对特定机器学习应用程序潜在的隐私侵犯的理解。项目团队在差异隐私、机器学习和网络安全方面拥有专业知识，并计划了一系列教育任务和外展活动：关于值得信赖的机器学习和隐私的公共课程材料，指导本科生和研究生进行研究项目，以及与行业合作伙伴进行合作。该项目包括三个相互关联的重点，解决隐私审计的不同方面。第一个重点是通过开发比以前的工作具有更强保证的最优成员推理攻击，为隐私审计奠定理论基础。第二部分介绍了审计凸机器学习模型和神经网络的新方法，通过使用对抗性机器学习中开发的中毒攻击的见解。最后的推力设计了用于审计在持续学习范式下训练的机器学习模型的端到端隐私泄漏的工具。这些研究重点为审计私有算法提供了一套技术，其目标是为从业者提供指导，指导他们如何选择私有算法及其参数，以平衡感兴趣任务的效用和隐私保证。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The use of statistics and machine learning to analyze sensitive data poses serious privacy risks to users who contribute their data to these algorithms, including: data reconstruction (revealing large portions of the training data), membership inference (revealing the presence of specific individuals in the training data), and data memorization (revealing specific training examples). Differential privacy has now become the standard for protecting data privacy in machine learning and statistics, as it offers a strong, formal, and quantitative guarantee of individual privacy. However, for many deployments of differential privacy there remains a significant gap between the formal guarantees and expectations in practice, and this project aims to bridge this gap by auditing deployed algorithms to discover and explain their privacy properties. The project’s novelties are building the theoretical foundations of privacy auditing and designing empirical methods that measure the privacy leakage of private algorithms in real-world scenarios. The project’s broader significance and importance will be in specific recommendations to practitioners on choice and use of private algorithms, and an understanding of the potential privacy violations for specific machine learning applications. The project team has expertise in differential privacy, machine learning, and cybersecurity, and plans a set of education tasks and outreach activities: public course materials on trustworthy machine learning and privacy, mentoring undergraduate and graduate students in research projects, and collaboration with industry partners. This project includes three interconnected thrusts addressing different aspects of privacy auditing. The first thrust lays the theoretical foundation of privacy auditing by developing optimal membership inference attacks with stronger guarantees than previous work. The second thrust introduces novel methods for auditing convex machine learning models and neural networks, by using insights from poisoning attacks developed in adversarial machine learning. The final thrust designs tools for auditing end-to-end privacy leakage of machine learning models trained under the continual-learning paradigm. These research thrusts enable a suite of techniques for auditing private algorithms, with the goal of providing guidance to practitioners on how to select private algorithms and their parameters to balance the utility and privacy guarantees on tasks of interest.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.