Impact of the Distribution Parameter of Data Sampling Approaches on Software Defect Prediction Models
Impact of the Distribution Parameter of Data Sampling Approaches on Software Defect Prediction Models
复制标题
DOI：
10.1109/apsec.2017.76
复制
发表时间：
2017-12
期刊：
影响因子：
--
通讯作者：
K. E. Bennin;J. Keung;Akito Monden
中科院分区：
文献类型：
--
作者：
K. E. Bennin;J. Keung;Akito Monden
已知采样方法会影响缺陷预测性能。这些采样方法具有可配置的参数，这些参数可以显著影响预测性能。然而，这是不切实际的，以评估的影响，所有可能的不同设置的参数空间的所有几个现有的采样方法。所有采样方法中存在的一个恒定且易于调整的参数是数据集中有缺陷和无缺陷模块的分布，称为Pfp（易出错模块的百分比）。在本文中，我们调查和评估的缺陷预测模型的性能，其中的采样方法的PFP参数进行了调整。对10个静态度量项目的20个版本的5个预测模型的7种采样方法的实证实验和评估表明：（1）调整Pfp参数后，接收者工作特征曲线（AUC）下的面积性能没有改善，（2）pf（假警报）性能随着Pfp的增加而下降。(3)很难在不同的PFP率上实现稳定的预测。因此，我们得出结论，Pfp参数设置可以对缺陷预测模型的性能（AUC除外）产生很大影响。因此，我们建议研究人员使用采样方法的Pfp参数进行实验，因为训练数据集的分布各不相同。
Sampling methods are known to impact defect prediction performance. These sampling methods have configurable parameters that can significantly affect the prediction performance. It is however, impractical to assess the effect of all the possible different settings in the parameter space for all the several existing sampling methods. A constant and easy to tweak parameter present in all sampling methods is the distribution of the defective and non-defective modules in the dataset known as Pfp (% of fault-prone modules). In this paper, we investigate and assess the performance of defect prediction models where the Pfp parameter of sampling methods are tweaked. An empirical experiment and assessment of seven sampling methods on five prediction models over 20 releases of 10 static metric projects indicate that (1) Area Under the Receiver Operating Characteristics Curve (AUC) performance is not improved after tweaking the Pfp parameter, (2) pf (false alarms) performance degrades as the Pfp is increased. (3) a stable predictor is difficult to achieve across different Pfp rates. Hence, we conclude that the Pfp parameter setting can have a large impact on the performance (except AUC) of defect prediction models. We thus recommend researchers experiment with the Pfp parameter of the sampling method since the distribution of training datasets vary.