CRII: SaTC: PrivateNet - Preserving Differential Privacy in Deep Learning under Model Attacks

机器学习在医疗保健领域的快速发展带来了明显的隐私问题，当深度神经网络和其他模型基于患者的个人和高度敏感的数据（如临床记录或跟踪的健康数据）建立时。此外，这些模型可能容易受到试图推断用于构建模型的敏感数据的攻击者的攻击。这提出了重要的研究问题，即如何开发机器学习模型，保护私人数据免受推理攻击，同时仍然是准确和有用的预测模型，以及重要的实际考虑因素，即患者数据的这些风险如何可能使医疗保健提供者面临基于HIPAA和相关法规的法律诉讼。为了解决这些问题，该项目将开发一个名为PrivateNet的框架，用于在模型攻击下保护深度神经网络的隐私，为深度学习中使用的数据提供强大的隐私保护。PrivateNet将在常用的机器学习框架之上开发，为项目的发现在工业和教育环境中产生影响提供了途径。该项目的一个关键要点是更好地理解和防御模型推理攻击，包括众所周知的基本模型攻击和通过经典机密性和完整性模型的棱镜开发的新攻击。通过对这些攻击的广泛分析，团队将了解学习方法的关键方面的相对风险。特别是，易受攻击的特征、参数和相关性，这些是进行模型攻击所必需的，将在基于差分隐私思想的新型威胁感知隐私保护方法中被自动识别和保护。具体来说，该团队将开发自适应隐私保护机制，在学习过程中最脆弱的方面分配噪声，在深度学习模型中提供强大的差异隐私保护，同时保持高模型效用。该项目有望为关键的隐私保护技术奠定基础，以保护用户在模型攻击下的个人和高度敏感的深度学习数据。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The rapid development of machine learning in the domain of healthcare presents clear privacy issues, when deep neural networks and other models are built based on patients' personal and highly sensitive data such as clinical records or tracked health data. Further, these models can be vulnerable to attackers trying to infer the sensitive data that was used to build the model. This raises important research questions about how to develop machine learning models that protect private data against inference attacks while still being accurate and useful predictive models, as well as important practical considerations about how these risks to patient data may expose health care providers to legal action based on HIPAA and related regulations. To address these questions, this project will develop a framework, called PrivateNet, for privacy preservation in deep neural networks under model attacks to offer strong privacy protections for data used in deep learning. PrivateNet will be developed on top of commonly used machine learning frameworks, providing ways for the project's findings to have impact in both industry and educational contexts.A key thrust of the project is to better understand and defend against model inference attacks, including both well-known fundamental model attacks and novel attacks developed through prism of the classical confidentiality and integrity models. Through an extensive analysis of these attacks, the team will develop an understanding of the relative risks of key aspects of learning approaches. In particular, vulnerable features, parameters, and correlations, which are essential to conduct model attacks, will be automatically identified and protected in a novel threat-aware privacy preserving approach based on ideas from differential privacy. Specifically, the team will develop adaptive privacy preserving mechanisms that distribute noise across the most vulnerable aspects of the learning process to provide strong differential privacy protections in deep learning models while maintaining high model utility. The project is expected to lay a foundation of key privacy-preserving techniques to protect users' personal and highly sensitive data in deep learning under model attacks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.