Measuring and Modeling the Label Dynamics of Online Anti-Malware Engines
Measuring and Modeling the Label Dynamics of Online Anti-Malware Engines
复制标题
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Shuofei Zhu;J. Shi;Limin Yang;Boqin Qin;Ziyi Zhang;Linhai Song;Gang Wang
中科院分区：
文献类型：
--
作者：
Shuofei Zhu;J. Shi;Limin Yang;Boqin Qin;Ziyi Zhang;Linhai Song;Gang Wang
VirusTotal提供来自大量反恶意软件引擎的恶意软件标签，并被研究人员大量用于恶意软件注释和系统评估。由于不同的引擎经常彼此不一致，研究人员使用了各种方法来聚合它们的标签。在本文中，我们采用数据驱动的方法对研究人员使用的常见标签方法进行分类、推理和验证。我们首先fi调查了115篇使用VirusTotal的学术论文，并确定了常见的方法。然后，我们从65个VirusTotal引擎收集了一年内超过14,000个fiLE(包括手动验证的fi基本事实的子集)的VirusTotal标签的每日快照。我们的分析验证了基于阈值的标签聚合在稳定fiLES标签方面的好处，并指出了阈值选择不当的影响。我们表明，精心挑选的“可信”引擎并不总是表现良好，某些引擎组具有很强的相关性，不应单独对待。最后，我们的经验表明，某些引擎无法对提交的fiLES进行深入分析，很容易产生误报。基于我们的fi代码，我们为将来使用VirusTotal进行数据注释提供了建议。
VirusTotal provides malware labels from a large set of anti-malware engines, and is heavily used by researchers for malware annotation and system evaluation. Since different engines often disagree with each other, researchers have used various methods to aggregate their labels. In this paper, we take a data-driven approach to categorize, reason, and validate common labeling methods used by researchers. We first survey 115 academic papers that use VirusTotal, and identify common methodologies. Then we collect the daily snapshots of VirusTotal labels for more than 14,000 files (including a subset of manually verified ground-truth) from 65 VirusTotal engines over a year. Our analysis validates the benefits of threshold-based label aggregation in stabilizing files’ labels, and also points out the impact of poorly-chosen thresholds. We show that hand-picked “trusted” engines do not always perform well, and certain groups of engines are strongly correlated and should not be treated independently. Finally, we empirically show certain engines fail to perform in-depth analysis on submitted files and can easily produce false positives. Based on our findings, we offer suggestions for future usage of VirusTotal for data annotation.