Big Data Analytics for Intrusion Detection System: Statistical Decision-Making Using Finite Dirichlet Mixture Models
Big Data Analytics for Intrusion Detection System: Statistical Decision-Making Using Finite Dirichlet Mixture Models
复制标题
DOI：
10.1007/978-3-319-59439-2_5
复制
发表时间：
2017-01-01
期刊：
影响因子：
--
通讯作者：
Slay, Jill
中科院分区：
文献类型：
--
作者：
Moustafa, Nour;Creech, Gideon;Slay, Jill
入侵检测系统已经成为检测网络领域各种恶意活动的重要机制。然而，在检测零日攻击时，该系统仍然面临着一个重要的限制，涉及到降低相对较高的误警率。因此，有必要不再孤立地考虑监测和分析网络数据的任务，而是将其与识别异常事件的决策方法进行优化整合。本章提出了一个可扩展的框架，用于构建一个有效的、轻量级的异常检测系统。该框架包括捕获和记录、预处理三个模块和一种新的统计决策引擎，称为基于Dirichlet混合模型的异常检测技术。第一个模块嗅探和收集网络数据，第二个模块分析和过滤这些数据，以提高决策引擎的性能。最后，设计了基于Dirichlet混合模型的决策引擎，将上下四分位数区间作为决策引擎。在两个著名的数据集NSL-KDD和UNSW-NB15上对该框架的性能进行了评估。实证结果表明，对网络数据的统计分析有助于选择最适合网络数据的最优模型。此外，基于Dirichlet混合模型的异常检测技术比其他三种引人注目的技术具有更高的检测率和更低的虚警率。这些技术建立在相关性和距离度量的基础上，无法检测到模仿正常活动的现代攻击，而所提出的技术是使用Dirichlet混合模型和四分位数范围的精确边界来发现合法向量和攻击向量之间的微小差异，从而有效地识别这些攻击。
An intrusion detection system has become a vital mechanism to detect a wide variety of malicious activities in the cyber domain. However, this system still faces an important limitation when it comes to detecting zero-day attacks, concerning the reduction of relatively high false alarm rates. It is thus necessary to no longer consider the tasks of monitoring and analysing network data in isolation, but instead optimise their integration with decision-making methods for identifying anomalous events. This chapter presents a scalable framework for building an effective and lightweight anomaly detection system. This framework includes three modules of capturing and logging, pre-processing and a new statistical decision engine, called the Dirichlet mixture model based anomaly detection technique. The first module sniffs and collects network data while the second module analyses and filters these data to improve the performance of the decision engine. Finally, the decision engine is designed based on the Dirichlet mixture model with a lower-upper interquartile range as decision engine. The performance of this framework is evaluated on two well-known datasets, the NSL-KDD and UNSW-NB15. The empirical results showed that the statistical analysis of network data helps in choosing the best model which correctly fits the network data. Additionally, the Dirichlet mixture model based anomaly detection technique provides a higher detection rate and lower false alarm rate than other three compelling techniques. These techniques were built based on correlation and distance measures that cannot detect modern attacks which mimic normal activities, whereas the proposed technique was established using the Dirichlet mixture model and precise boundaries of interquartile range for finding small differences between legitimate and attack vectors, efficiently identifying these attacks.