SaTC: CORE: Small: Collaborative: When Adversarial Learning Meets Differential Privacy: Theoretical Foundation and Applications

机器学习的普及暴露了软件系统中新的严重漏洞，在软件系统中，部署的深度神经网络可被利用来揭示私人训练数据中的敏感信息，并使模型错误分类。然而，现有的学习算法在理论和实践上都没有被设计成对这种隐私和完整性攻击同时具有健壮性。在现场试验中，这种缺乏保护和有效性的情况会显著降低基于机器学习的系统的性能，并使敏感数据面临高风险，从而使服务提供商面临基于HIPAA/HITECH法律和相关法规的法律诉讼。该项目旨在开发第一个框架来推进和无缝集成关键技术，包括对抗性学习、隐私保护和认证防御，提供针对隐私和完整性攻击的严密和可靠的保护，同时保持深度神经网络的高模型实用性。该系统正在为可扩展的、复杂的和常用的机器学习框架而开发，为行业和教育环境提供根本影响。该项目的最终目标是构建对抗性学习中隐私保护的核心基础，以更好地解决模型效用、隐私损失和认证防御之间的权衡。因此，该团队通过引入一套新的严格理论来解决模型效用和隐私损失之间的权衡，从理论上将对抗性学习和隐私保护联系起来。为了进一步加强系统的安全性，该小组将进行一类新的攻击，以发现以前未知和未受保护的漏洞，包括数据实例之间高度敏感和隐藏的关联结构，这些结构将被用来放大现有的模型攻击。在这一努力的基础上，将自动识别和保护易受攻击的特征和相关性，从而在给定模型训练和推理的情况下，实现统一的健壮和隐私保护学习。最后，该团队将优化模型效用、隐私损失和认证防御之间的权衡。该项目预计将为关键隐私保护技术奠定理论和实践基础，以保护用户在模型攻击下的对抗性学习中的个人和高度敏感数据。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The pervasiveness of machine learning exposes new and severe vulnerabilities in software systems, where deployed deep neural networks can be exploited to reveal sensitive information in private training data, and to make the models misclassify. However, existing learning algorithms have not been designed to be simultaneously robust to such privacy and integrity attacks, in both theory and practice. In field trials, such lack of protection and efficacy significantly degrades the performance of machine learning-based systems, and puts sensitive data at high risk, thereby exposing service providers to legal action based on HIPAA/HITECH law and related regulations. This project aims to develop the first framework to advance and seamlessly integrate key techniques, including adversarial learning, privacy preserving, and certified defenses, offering tight and reliable protection against both privacy and integrity attacks, while retaining high model utility in deep neural networks. The system is being developed for scalable, complex, and commonly used machine learning frameworks, providing a fundamental impact to both industry and educational environments.An ultimate goal of this project is to build a core foundation of privacy preservation in adversarial learning, to better address the trade-off between model utility, privacy loss, and certified defenses. Accordingly, the team theoretically connects adversarial learning and privacy preservation by introducing a new set of rigorous theories to address the trade-off between model utility and privacy loss. To further strengthen the safety of the system, the team will conduct a new class of attacks towards discovering previously unknown and unprotected vulnerabilities, including highly sensitive and hidden correlation structures among data instances, which will be used to amplify existing model attacks. Based upon that effort, vulnerable features and correlations will be automatically identified and protected, towards unified robust and privacy preserving learning, given both model training and inference. Finally, the team will optimize the trade-off among model utility, privacy loss, and certified defenses. The project is expected to lay a theoretical and practical foundation of key privacy-preserving techniques to protect users' personal and highly sensitive data in adversarial learning under model attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.