Measuring Relative Accuracy of Malware Detectors in the Absence of Ground Truth
Measuring Relative Accuracy of Malware Detectors in the Absence of Ground Truth
复制标题
DOI：
10.1109/milcom.2018.8599730
复制
发表时间：
2018-10
期刊：
影响因子：
--
通讯作者：
John Charlton;Pang Du;Jin-Hee Cho;Shouhuai Xu
中科院分区：
文献类型：
--
作者：
John Charlton;Pang Du;Jin-Hee Cho;Shouhuai Xu
在本文中，我们在没有关于恶意软件检测器质量的地面实况的情况下测量恶意软件检测器的相对准确性（即，检测精度）或样本文件的类别，（即，恶意的或良性的）。特别是，我们感兴趣的是在没有实际检测质量的地面真相的情况下测量恶意软件检测器的顺序规模。为此，我们提出了一种算法来估计恶意软件检测器的相对准确性。基于合成数据与已知的地面真相，我们的特点时，所提出的算法导致准确地估计恶意软件检测器的相对精度。我们使用我们提出的算法的基础上的真实的数据集，包括1070万个文件和62个恶意软件检测器，从VirusTotal的测量相对准确性的真实世界的恶意软件检测器。
In this paper, we measure the relative accuracy of malware detectors in the absence of ground truth regarding the quality of malware detectors (i.e., the detection accuracy) or the class of sample files, (i.e., malicious or benign). In particular, we are interested in measuring the ordinal scale of mal ware detectors in the absence of the ground truth of their actual detection quality. To this end, we propose an algorithm to estimate the relative accuracy of the malware detectors. Based on synthetic data with known ground truth, we characterize when the proposed algorithm leads to accurately estimating the relative accuracy of the malware detectors. We show the measured relative accuracy of real-world malware detectors using our proposed algorithm based on a real dataset consisting of 10.7 million files and 62 malware detectors, obtained from VirusTotal.