Statistical Estimation of Malware Detection Metrics in the Absence of Ground Truth
Statistical Estimation of Malware Detection Metrics in the Absence of Ground Truth
复制标题
DOI：
10.1109/tifs.2018.2833292
复制
发表时间：
2018-05
影响因子：
6.8
通讯作者：
Pang Du;Zheyuan Sun;Huashan Chen;Jin-Hee Cho;Shouhuai Xu
中科院分区：
文献类型：
--
作者：
Pang Du;Zheyuan Sun;Huashan Chen;Jin-Hee Cho;Shouhuai Xu
安全度量的准确测量是一个关键的研究问题，因为不正确或不准确的测量过程可能会破坏度量的有用性。这是一个非常具有挑战性的问题，特别是当地面真相是未知的或嘈杂的。在本文中，我们测量五个恶意软件检测指标的情况下，地面真相，这是一个现实的设置，提出了许多技术挑战。最终目标是开发原则性的、自动化的方法，以尽可能高的精度测量这些指标。这个问题自然要求调查统计估计铸造的测量问题作为一个统计估计问题。我们提出这五个恶意软件检测指标的统计估计。通过研究这些估计量的统计特性，我们描述了估计量何时是准确的，以及在什么情况下可以进行什么调整来改善它们。我们使用合成数据与已知的地面真相，以验证这些统计估计。然后，我们采用这些估计来衡量从VirusTotal收集的大数据集的五个指标。
The accurate measurement of security metrics is a critical research problem, because an improper or inaccurate measurement process can ruin the usefulness of the metrics. This is a highly challenging problem, particularly when the ground truth is unknown or noisy. In this paper, we measure five malware detection metrics in the absence of ground truth, which is a realistic setting that imposes many technical challenges. The ultimate goal is to develop principled, automated methods for measuring these metrics at the maximum accuracy possible. The problem naturally calls for investigations into statistical estimators by casting the measurement problem as a statistical estimation problem. We propose statistical estimators for these five malware detection metrics. By investigating the statistical properties of these estimators, we characterize when the estimators are accurate, and what adjustments can be made to improve them under what circumstances. We use synthetic data with known ground truth to validate these statistical estimators. Then, we employ these estimators to measure five metrics with respect to a large data set collected from VirusTotal.