SaTC: CORE: Small: Adversarial Learning via Modeling Interpretation

机器学习（ML）模型在社会中越来越重要，其应用包括恶意软件检测、在线内容过滤和排名以及自动驾驶汽车。然而，这些模型很容易受到攻击者的攻击，攻击者通过提交错误或操纵的数据来造成错误，从而对模型做出的决策以及依赖它们的系统和人员造成潜在的伤害。此外，许多常见的机器学习模型以人类难以理解的方式做出决策，这导致人们呼吁开发建模技术，使模型更具可解释性和可解释性。该项目位于对抗性和可解释ML的交叉点，其关键见解是，随着模型在其做出的个人决策和用于区分不同决策的规则方面变得更加可解释，这种可解释性可能会提供可用于创建和防御对抗性攻击的额外信息。 项目的总体目标是测试这种洞察力，并通过开发一个对抗性学习框架来为安全和数据挖掘社区做出贡献，该框架利用ML模型和结果的可解释性来识别和减轻对抗性攻击的风险，特别是在大数据的背景下。该项目还包含一个重要的教育部分，包括将研究纳入课程编制，并为本科生和代表性不足的学生提供研究机会。首先是通过从实例级、类级和特定的一组深度神经网络三个方面分析建模解释来制定有效的攻击策略。这使得通过理解ML模型的底层工作机制来发起更有效的攻击成为可能。第二个重点是开发防御策略，以提高ML模型对这些对抗性攻击的鲁棒性。所提出的防御策略针对典型知识发现管道中的三个主要步骤，包括训练数据细化、模型架构修改和测试数据过滤。虽然现有的努力是基于不断探测构建的系统，并在发现预测错误后更新模型参数，但所提出的工作提供了一种主动解决问题的方法。第三个重点是开发对抗性学习算法，以应对挑战并利用大数据带来的机遇。具体来说，开发的对抗性攻击和防御算法将处理大规模、异构和关系数据。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning (ML) models are increasingly important in society, with applications including malware detection, online content filtering and ranking, and self-driving cars. However, these models are vulnerable to adversaries attacking them by submitting incorrect or manipulated data with the goal of causing errors, causing potential harm to both the decisions the models make and the systems and people who rely on them. Further, many common ML models make decisions in ways that are hard for humans to understand, leading to calls to develop modeling techniques that make the models more explainable and interpretable. This project sits at the intersection of adversarial and explainable ML, with the key insight that as models become more interpretable in terms of both the individual decisions they make and the rules they use to distinguish between different decisions, this interpretability will likely provide additional information that can be used to both create and defend against adversarial attacks. The overall project goal is to test this insight and contribute to both the security and data mining communities by developing an adversarial learning framework that leverages interpretability of ML models and results to both identify and mitigate the risks of adversarial attacks, especially in the context of big data. The project also contains a significant educational component, including incorporating the research into curriculum development and providing research opportunities to undergraduate and underrepresented students.The project consists of three research thrusts. The first is to develop effective attacking strategies by analyzing modeling interpretation from three aspects including instance level, class level, and a specific group of deep neural networks. This enables more effective attacks to be initiated through understanding the underlying working mechanisms of ML models. The second thrust is to focus on developing defensive strategies to improve the robustness of ML models against these adversarial attacks. The proposed defensive strategies are aimed at the three major steps in a typical knowledge discovery pipeline including training data refinement, model architecture modification, and test data filtering. While existing efforts are based on continuously probing built systems and updating model parameters once prediction mistakes are discovered, the proposed work provides a proactive way to tackle the problem. The third thrust is to develop adversarial learning algorithms to deal with challenges and take advantage of opportunities brought by big data. Specifically, the developed adversarial attacking and defensive algorithms will deal with large-scale, heterogeneous, and relational data. This will enable the proposed algorithms to scale to real-world applications demonstrating challenging data characteristics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.