An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security Needs
An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security Needs
复制标题
DOI：
10.1109/access.2020.2969780
复制
发表时间：
2020-01-01
期刊：
影响因子：
3.9
通讯作者：
Verma, Rakesh M.
中科院分区：
文献类型：
--
作者：
El Aassal, Ayman;Baki, Shahryar;Verma, Rakesh M.
我们在多样且广泛的数据集上对网络钓鱼特征进行了深入、系统的基准测试研究和评估。我们根据每个特征的解释和用途提出了一种新的特征分类法。接下来，我们提出了一个名为“PhishBench”的基准测试框架，它使我们能够在相同的实验条件下，即统一的系统规格、数据集、分类器和评估指标下，系统而全面地评估和比较现有的网络钓鱼检测特征。PhishBench在网络钓鱼相关研究的基准测试领域是首创的，它包含了全面且系统的评估以及特征比较。我们使用PhishBench在新的多样数据集上测试网络钓鱼文献中发表的方法，以检验它们的稳健性和可扩展性。我们研究数据集特征，例如合法与网络钓鱼的比例变化以及不平衡数据集规模的增加，如何影响分类性能。我们的结果表明，网络钓鱼攻击的不平衡性质会影响检测系统的性能，研究人员在提出新方法时应考虑到这一点。我们还发现，仅仅重新训练不足以抵御新的攻击。需要新的特征和技术来阻止攻击者愚弄检测系统。
We perform an in-depth, systematic benchmarking study and evaluation of phishing features on diverse and extensive datasets. We propose a new taxonomy of features based on the interpretation and purpose of each feature. Next, we propose a benchmarking framework called 'PhishBench,' which enables us to evaluate and compare the existing features for phishing detection systematically and thoroughly under identical experimental conditions, i.e., unified system specification, datasets, classifiers, and evaluation metrics. PhishBench is a first in the field of benchmarking phishing related research and incorporates thorough and systematic evaluation and feature comparison. We use PhishBench to test methods published in the phishing literature on new and diverse datasets to check their robustness and scalability. We study how dataset characteristics, e.g., varying legitimate to phishing ratios and increasing the size of imbalanced datasets, affect classification performance. Our results show that the imbalanced nature of phishing attacks affects the detection systems' performance and researchers should take this into account when proposing a new method. We also found that retraining alone is not enough to defeat new attacks. New features and techniques are required to stop attackers from fooling detection systems.