Software defect prediction: do different classifiers find the same defects?
Software defect prediction: do different classifiers find the same defects?
复制标题
DOI：
10.1007/s11219-016-9353-3
复制
发表时间：
2018-06-01
影响因子：
1.9
通讯作者：
Petric, Jean
中科院分区：
文献类型：
--
作者：
Bowes, David;Hall, Tracy;Petric, Jean
在过去的10年中，已经发布了数百种不同的缺陷预测模型。据报道，这些模型中使用的分类器的性能与大约80％召回的预测性能上限的模型相似。我们研究了四个分类器预测和分析这些分类器产生的预测不确定性水平的个体缺陷。我们执行灵敏度分析，以比较随机森林，Na <Ve贝叶斯，RPART和SVM分类器的性能，以预测NASA，开源和商业数据集中的缺陷。每个分类器制作的缺陷预测都在混淆矩阵中捕获，并比较每个分类器的预测不确定性。尽管这四个分类器对这四个分类器进行了相似的预测性能值，但每个分类器都检测到不同的缺陷集。某些分类器在预测缺陷方面比其他分类器更一致。我们的结果证实，特定分类器可以检测到缺陷的独特子集。但是，尽管某些分类器在他们做出的预测中保持一致，但其他分类器的预测却有所不同。鉴于我们的结果，我们得出的结论是，分类器合奏具有并非基于多数投票的决策策略，可能在缺陷预测中表现最佳。
During the last 10 years, hundreds of different defect prediction models have been published. The performance of the classifiers used in these models is reported to be similar with models rarely performing above the predictive performance ceiling of about 80% recall. We investigate the individual defects that four classifiers predict and analyse the level of prediction uncertainty produced by these classifiers. We perform a sensitivity analysis to compare the performance of Random Forest, Na < ve Bayes, RPart and SVM classifiers when predicting defects in NASA, open source and commercial datasets. The defect predictions that each classifier makes is captured in a confusion matrix and the prediction uncertainty of each classifier is compared. Despite similar predictive performance values for these four classifiers, each detects different sets of defects. Some classifiers are more consistent in predicting defects than others. Our results confirm that a unique subset of defects can be detected by specific classifiers. However, while some classifiers are consistent in the predictions they make, other classifiers vary in their predictions. Given our results, we conclude that classifier ensembles with decision-making strategies not based on majority voting are likely to perform best in defect prediction.