XAdv: Robust Explanations for Malware Detection

恶意软件（简称“恶意软件”）是指任何执行恶意活动的软件，例如窃取信息（例如间谍软件）和破坏系统（例如勒索软件）。恶意软件作者不断更新他们的攻击策略以逃避反病毒系统的检测，并自动生成比原始恶意软件更难识别的同一恶意软件的多个变体。传统的恶意软件检测方法依赖于手动定义的模式（例如，字节序列），既耗时又容易出错。因此，学术界和行业研究人员已经开始探索机器学习（ML）如何帮助检测新的、看不见的恶意软件类型。在这种情况下，解释ML决策对于安全分析人员验证某个决策的正确性以及更快地开发补丁和补救措施至关重要。然而，已经证明攻击者可以在ML系统中诱导任意错误的解释；这是通过仔细修改恶意软件的几个字节来实现的。这个项目，XAdv（“X”代表解释，“Adv”代表对抗鲁棒性），旨在为恶意软件检测设计“鲁棒性解释”，即对模型决策的解释，对于安全分析师来说，这是易于理解和可视化的（以支持更快的恶意验证和补丁开发），并且即使存在恶意软件随着时间的推移而进化和规避恶意软件作者，这也是值得信赖和可靠的。此外，该项目将探索如何使用鲁棒解释来自动适应基于ml的恶意软件检测模型，以适应新的威胁，以及集成来自安全分析师的鲁棒解释反馈的领域知识，以提高检测准确性。

Malware (short for "malicious software") refers to any software that perform malicious activities, such as stealing information (e.g., spyware) and damaging systems (e.g., ransomware). Malware authors constantly update their attack strategies to evade detection of antivirus systems, and automatically generate multiple variants of the same malware that are harder to recognize than the original. Traditional malware detection methods relying on manually defined patterns (e.g., sequences of bytes) are time consuming and error prone. Hence, academic and industry researchers have started exploring how Machine Learning (ML) can help in detecting new, unseen malware types. In this context, explaining ML decisions is fundamental for security analysts to verify correctness of a certain decision, and develop patches and remediations faster. However, it has been shown that attackers can induce arbitrary, wrong explanations in ML systems; this is achieved by carefully modifying a few bytes of their malware. This project, XAdv ("X" for explanation, and "Adv" for adversarial robustness), aims to design "robust explanations" for malware detection, i.e., explanations of model decisions which are easy to understand and visualize for security analysts (to support faster verification of maliciousness, and development of patches), and which are trustworthy and reliable even in presence of malware evolution over time and evasive malware authors. Moreover, this project will explore how robust explanations can be used to automatically adapt ML-based malware detection models to new threats over time, as well as to integrate domain knowledge from security analysts' feedback from robust explanations to improve detection accuracy.