SaTC: CORE: Small: Adversarial Learning via Modeling Interpretation

SaTC：核心：小：通过建模解释进行对抗性学习

• 批准号: 1816497
• 负责人: Guofei Gu
• 金额: $ 50万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816497&HistoricalAwards=false
• 关键词: SaTC; CORE; Small; Adversarial; Learning

Machine learning (ML) models are increasingly important in society, with applications including malware detection, online content filtering and ranking, and self-driving cars. However, these models are vulnerable to adversaries attacking them by submitting incorrect or manipulated data with the goal of causing errors, causing potential harm to both the decisions the models make and the systems and people who rely on them. Further, many common ML models make decisions in ways that are hard for humans to understand, leading to calls to develop modeling techniques that make the models more explainable and interpretable. This project sits at the intersection of adversarial and explainable ML, with the key insight that as models become more interpretable in terms of both the individual decisions they make and the rules they use to distinguish between different decisions, this interpretability will likely provide additional information that can be used to both create and defend against adversarial attacks. The overall project goal is to test this insight and contribute to both the security and data mining communities by developing an adversarial learning framework that leverages interpretability of ML models and results to both identify and mitigate the risks of adversarial attacks, especially in the context of big data. The project also contains a significant educational component, including incorporating the research into curriculum development and providing research opportunities to undergraduate and underrepresented students.The project consists of three research thrusts. The first is to develop effective attacking strategies by analyzing modeling interpretation from three aspects including instance level, class level, and a specific group of deep neural networks. This enables more effective attacks to be initiated through understanding the underlying working mechanisms of ML models. The second thrust is to focus on developing defensive strategies to improve the robustness of ML models against these adversarial attacks. The proposed defensive strategies are aimed at the three major steps in a typical knowledge discovery pipeline including training data refinement, model architecture modification, and test data filtering. While existing efforts are based on continuously probing built systems and updating model parameters once prediction mistakes are discovered, the proposed work provides a proactive way to tackle the problem. The third thrust is to develop adversarial learning algorithms to deal with challenges and take advantage of opportunities brought by big data. Specifically, the developed adversarial attacking and defensive algorithms will deal with large-scale, heterogeneous, and relational data. This will enable the proposed algorithms to scale to real-world applications demonstrating challenging data characteristics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习(ML)模型在社会中越来越重要，应用程序包括恶意软件检测、在线内容过滤和排名以及自动驾驶汽车。然而，这些模型很容易受到攻击者的攻击，他们提交不正确或被篡改的数据，目的是造成错误，从而对模型做出的决策以及依赖它们的系统和人员造成潜在损害。此外，许多常见的ML模型以人类难以理解的方式进行决策，这导致了开发建模技术的呼声，使模型更易于解释和解释。这个项目位于对抗性和可解释性ML的交叉点上，关键的洞察力是，随着模型在他们做出的个人决策和他们用来区分不同决策的规则方面变得更具解释性，这种可解释性可能会提供可用于创建和防御对手攻击的额外信息。项目的总体目标是测试这一洞察力，并通过开发一个对抗性学习框架来为安全和数据挖掘社区做出贡献，该框架利用ML模型和结果的可解释性来识别和降低对抗性攻击的风险，特别是在大数据的背景下。该项目还包含一个重要的教育部分，包括将研究纳入课程发展，并为本科生和代表性不足的学生提供研究机会。该项目包括三个研究推动力。首先是从实例级、类级和特定的一组深层神经网络三个方面分析建模解释，从而制定有效的攻击策略。这使得通过了解ML模型的底层工作机制来发起更有效的攻击。第二个重点是专注于开发防御策略，以提高ML模型对这些对抗性攻击的健壮性。所提出的防御策略针对典型知识发现管道中的三个主要步骤，包括训练数据精化、模型体系结构修改和测试数据过滤。虽然现有的工作是基于不断探测建立的系统并在发现预测错误后更新模型参数，但拟议的工作提供了一种主动解决问题的方法。第三个主旨是开发对抗性学习算法，以应对挑战并利用大数据带来的机遇。具体地说，开发的对抗性攻击和防御算法将处理大规模、异质和关系数据。这将使建议的算法能够扩展到展示具有挑战性的数据特征的真实世界应用程序。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Life after Speech Recognition: Fuzzing Semantic Misinterpretation for Voice Assistant Applications

• DOI: 10.14722/ndss.2019.23525
• 发表时间: 2019
• 期刊: Proceedings 2019 Network and Distributed System Security Symposium
• 作者: Yangyong Zhang;Lei Xu;Abner Mendoza;Guangliang Yang;Phakpoom Chinprutthiwong;G. Gu