SaTC: CORE: Small: Adversarial Learning via Modeling Interpretation

SaTC：核心：小：通过建模解释进行对抗性学习

• 批准号: 1816497
• 负责人: Guofei Gu
• 金额: $ 50万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816497&HistoricalAwards=false
• 关键词: SaTC; CORE; Small; Adversarial; Learning

Machine learning (ML) models are increasingly important in society, with applications including malware detection, online content filtering and ranking, and self-driving cars. However, these models are vulnerable to adversaries attacking them by submitting incorrect or manipulated data with the goal of causing errors, causing potential harm to both the decisions the models make and the systems and people who rely on them. Further, many common ML models make decisions in ways that are hard for humans to understand, leading to calls to develop modeling techniques that make the models more explainable and interpretable. This project sits at the intersection of adversarial and explainable ML, with the key insight that as models become more interpretable in terms of both the individual decisions they make and the rules they use to distinguish between different decisions, this interpretability will likely provide additional information that can be used to both create and defend against adversarial attacks. The overall project goal is to test this insight and contribute to both the security and data mining communities by developing an adversarial learning framework that leverages interpretability of ML models and results to both identify and mitigate the risks of adversarial attacks, especially in the context of big data. The project also contains a significant educational component, including incorporating the research into curriculum development and providing research opportunities to undergraduate and underrepresented students.The project consists of three research thrusts. The first is to develop effective attacking strategies by analyzing modeling interpretation from three aspects including instance level, class level, and a specific group of deep neural networks. This enables more effective attacks to be initiated through understanding the underlying working mechanisms of ML models. The second thrust is to focus on developing defensive strategies to improve the robustness of ML models against these adversarial attacks. The proposed defensive strategies are aimed at the three major steps in a typical knowledge discovery pipeline including training data refinement, model architecture modification, and test data filtering. While existing efforts are based on continuously probing built systems and updating model parameters once prediction mistakes are discovered, the proposed work provides a proactive way to tackle the problem. The third thrust is to develop adversarial learning algorithms to deal with challenges and take advantage of opportunities brought by big data. Specifically, the developed adversarial attacking and defensive algorithms will deal with large-scale, heterogeneous, and relational data. This will enable the proposed algorithms to scale to real-world applications demonstrating challenging data characteristics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习（ML）模型在社会中越来越重要，其应用包括恶意软件检测、在线内容过滤和排名，以及自动驾驶汽车。然而，这些模型很容易受到攻击者的攻击，他们会提交不正确的或被操纵的数据，目的是导致错误，从而对模型做出的决策、系统和依赖模型的人造成潜在的危害。此外，许多常见的ML模型以人类难以理解的方式做出决策，这导致需要开发建模技术，使模型更具可解释性和可解释性。该项目位于对抗性和可解释性ML的交叉点，其关键见解是，随着模型在其做出的单个决策和用于区分不同决策的规则方面变得更具可解释性，这种可解释性可能会提供可用于创建和防御对抗性攻击的额外信息。整个项目的目标是测试这一见解，并通过开发一个对抗性学习框架来为安全和数据挖掘社区做出贡献，该框架利用ML模型和结果的可解释性来识别和减轻对抗性攻击的风险，特别是在大数据的背景下。该项目还包含一个重要的教育组成部分，包括将研究纳入课程开发，并为本科生和代表性不足的学生提供研究机会。该项目包括三个研究重点。首先，从实例级、类级和特定深度神经网络组三个方面分析建模解释，制定有效的攻击策略。通过理解ML模型的底层工作机制，可以发起更有效的攻击。第二个重点是专注于开发防御策略，以提高ML模型对这些对抗性攻击的鲁棒性。提出的防御策略针对典型知识发现管道中的三个主要步骤：训练数据细化、模型架构修改和测试数据过滤。虽然现有的工作是基于不断探测构建的系统，一旦发现预测错误就更新模型参数，但提出的工作提供了一种主动解决问题的方法。第三个重点是开发对抗性学习算法，以应对挑战并利用大数据带来的机遇。具体来说，所开发的对抗性攻击和防御算法将处理大规模、异构和关系数据。这将使所提出的算法能够扩展到展示具有挑战性的数据特征的实际应用中。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Life after Speech Recognition: Fuzzing Semantic Misinterpretation for Voice Assistant Applications

• DOI: 10.14722/ndss.2019.23525
• 发表时间: 2019
• 期刊: Proceedings 2019 Network and Distributed System Security Symposium
• 作者: Yangyong Zhang;Lei Xu;Abner Mendoza;Guangliang Yang;Phakpoom Chinprutthiwong;G. Gu