FastEmbed: Predicting vulnerability exploitation possibility based on ensemble machine learning algorithm
FastEmbed: Predicting vulnerability exploitation possibility based on ensemble machine learning algorithm
复制标题
DOI：
10.1371/journal.pone.0228439
复制
发表时间：
2020-02-06
期刊：
影响因子：
3.7
通讯作者：
Liu, Liang
中科院分区：
文献类型：
--
作者：
Fang, Yong;Liu, Yongcheng;Liu, Liang
近年来，发现并公开披露的漏洞数量呈急剧上升趋势。然而，考虑到只有一小部分漏洞被利用，利用漏洞的价值对攻击者来说各不相同。因此，快速排除不可利用的漏洞，并在有限的资源上实现最佳的补丁优先级已成为组织的当务之急。最近使用机器学习技术的工作通过从开源情报（OSINT）中提取特征来预测被利用的漏洞。然而，面对爆炸式增长的漏洞信息，将过去的方法应用于多种威胁情报还有改进的空间。需要一种更通用的方法来处理各种威胁情报源。而且，以往的方法都是采用传统的文本处理方法来处理与脆弱性相关的描述，只抓住了静态的统计特征，而忽略了文本的上下文和词语的含义。为了解决这些挑战，我们提出了一个利用预测模型，它是基于组合的fastText和LightGBM算法，称为fastEmbed。我们复制了最先进的漏洞利用预测工作的关键部分，并将其用作基准模型。我们的模型无论是在泛化能力还是在没有时间混杂的情况下的预测能力方面都优于基线模型，通过学习在极不平衡的数据集上嵌入与可扩展性相关的文本，平均整体改进为6.283%。此外，在预测野生环境中的漏洞利用方面，我们的模型也优于基线模型，少数类的F1测量值为0.586（比使用暗网/深网功能的工作提高了33.577%）。实验结果表明，该模型能够有效地提高对漏洞可利用性的描述能力和预测能力。
In recent years, the number of vulnerabilities discovered and publicly disclosed has shown a sharp upward trend. However, the value of exploitation of vulnerabilities varies for attackers, considering that only a small fraction of vulnerabilities are exploited. Therefore, the realization of quick exclusion of the non-exploitable vulnerabilities and optimal patch prioritization on limited resources has become imperative for organizations. Recent works using machine learning techniques predict exploited vulnerabilities by extracting features from open-source intelligence (OSINT). However, in the face of explosive growth of vulnerability information, there is room for improvement in the application of past methods to multiple threat intelligence. A more general method is needed to deal with various threat intelligence sources. Moreover, in previous methods, traditional text processing methods were used to deal with vulnerability related descriptions, which only grasped the static statistical characteristics but ignored the context and the meaning of the words of the text. To address these challenges, we propose an exploit prediction model, which is based on a combination of fastText and LightGBM algorithm and called fastEmbed. We replicate key portions of the state-of-the-art work of exploit prediction and use them as benchmark models. Our model outperforms the baseline model whether in terms of the generalization ability or the prediction ability without temporal intermixing with an average overall improvement of 6.283% by learning the embedding of vulnerability-related text on extremely imbalanced data sets. Besides, in terms of predicting the exploits in the wild, our model also outperforms the baseline model with an F1 measure of 0.586 on the minority class (33.577% improvement over the work using features from darkweb/deepweb). The results demonstrate that the model can improve the ability to describe the exploitability of vulnerabilities and predict exploits in the wild effectively.