权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

SaTC: CORE: Small: Detecting and Mitigating Unintended Learning

SaTC：核心：小：检测和减轻意外学习

基本信息

批准号：
1916717
负责人：
Vitaly Shmatikov
金额：
$ 50万
依托单位：
Cornell University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1916717&HistoricalAwards=false
关键词：
SaTC CORE Small Detecting Mitigating

项目摘要

Machine learning is fueling major advances in biomedical research, natural language processing, image recognition, self-driving vehicles, etc. These advances depend on the continuing availability of data. By assuring the integrity and privacy of both the data and the machine learning models based on this data, this project aims to bring the benefits of machine learning to all data holders. However, machine learning can unintentionally reveal sensitive data such as the identity of specific persons. This project will investigate unintended learning, i.e., what machine learning models are discovering beyond their stated objectives, and its consequences for the data on which the models are trained and to which they are applied. In this project, the first focus area is developing inference techniques that detect leakage of sensitive data and, more generally, determine what models are actually learning. This research will help identify the root causes of training-data memorization in deep models, develop methodology for detecting and measuring it, and help prevent deep models from unintentionally learning privacy-sensitive features of the data. The second focus area is developing and analyzing methods for mitigating unintended learning and ensuring that models do not contain unwanted or malicious functionality. In addition to improving privacy of the training and test data, this research will help detect and prevent backdoors in models trained on smartphones and other edge devices. All technologies developed as part of this project will be evaluated on state-of-the-art image-analysis and natural-language models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习正在推动生物医学研究、自然语言处理、图像识别、自动驾驶汽车等领域取得重大进展。这些进步取决于数据的持续可用性。通过确保数据和基于这些数据的机器学习模型的完整性和隐私性，该项目旨在将机器学习的好处带给所有数据持有者。然而，机器学习可能会无意中泄露敏感数据，例如特定人员的身份。该项目将研究意外学习，即机器学习模型在其既定目标之外发现了什么，以及它对模型所训练和应用的数据的影响。在这个项目中，第一个重点领域是开发推理技术，以检测敏感数据的泄漏，更一般地说，确定模型实际在学习什么。这项研究将有助于确定深度模型中训练数据记忆的根本原因，开发检测和测量方法，并有助于防止深度模型无意中学习数据的隐私敏感特征。第二个重点领域是开发和分析减轻意外学习的方法，并确保模型不包含不需要的或恶意的功能。除了提高训练和测试数据的隐私性外，这项研究还将有助于检测和防止在智能手机和其他边缘设备上训练的模型中的后门。作为该项目的一部分开发的所有技术将在最先进的图像分析和自然语言模型上进行评估。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（9）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Adversarial Semantic Collisions

DOI：
10.18653/v1/2020.emnlp-main.344
发表时间：
2020-11
期刊：
影响因子：
0
作者：
Congzheng Song;Alexander M. Rush;Vitaly Shmatikov
通讯作者：
Congzheng Song;Alexander M. Rush;Vitaly Shmatikov

De-Anonymizing Text by Fingerprinting Language Generation

DOI：
发表时间：
2020-06
期刊：
ArXiv
影响因子：
0
作者：
Zhen Sun;R. Schuster;Vitaly Shmatikov
通讯作者：
Zhen Sun;R. Schuster;Vitaly Shmatikov

Overlearning Reveals Sensitive Attributes

DOI：
发表时间：
2019-05
期刊：
ArXiv
影响因子：
0
作者：
Congzheng Song;Vitaly Shmatikov
通讯作者：
Congzheng Song;Vitaly Shmatikov

Auditing Data Provenance in Text-Generation Models

DOI：
10.1145/3292500.3330885
发表时间：
2018-11
期刊：
Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
影响因子：
0
作者：
Congzheng Song;Vitaly Shmatikov
通讯作者：
Congzheng Song;Vitaly Shmatikov

Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures