Improvement and Evaluation of Data Consistency Metric CIL for Software Engineering Data Sets
Improvement and Evaluation of Data Consistency Metric CIL for Software Engineering Data Sets
复制标题
软件工程数据集数据一致性度量CIL的改进与评估
DOI：
10.1109/access.2022.3188246
复制
发表时间：
2022
期刊：
影响因子：
3.9
通讯作者：
Akito Monden
中科院分区：
文献类型：
--
作者：
Maohua Gan;Zeynep Yucel;Akito Monden
源自实际软件产品及其开发过程的软件数据集广泛用于项目规划、管理、质量保证和过程改进等。尽管已证明某些数据集不适合这些目的，但数据集的数据质量通常不会在使用前进行评估。其主要原因是没有太多的度量标准来量化软件开发数据的适应性。在这方面，本研究试图通过设计一种新的和有效的数据质量评估方法来填补文献中的空白。为此，我们从案例不一致性级别（CIL）开始，它计算数据集中不一致项目对的数量以评估其一致性。基于一个大样本集的后续评估，我们描述了CIL是不是有效的，在某些数据集的质量评估。通过研究与CIL相关的问题，并消除它们，我们提出了一个改进的度量称为相似情况不一致性水平（SCIL）。我们的实证评估与54个数据样本来自6个大型项目的数据集表明，SCIL可以区分一致和不一致的数据集，和预测模型的软件开发工作量和生产力建立一致的数据集确实实现了相对较高的准确性。
Software data sets derived from actual software products and their development processes are widely used for project planning, management, quality assurance and process improvement, etc. Although it is demonstrated that certain data sets are not fit for these purposes, the data quality of data sets is often not assessed before using them. The principal reason for this is that there are not many metrics quantifying fitness of software development data. In that respect, this study makes an effort to fill in the void in literature by devising a new and efficient assessment method of data quality. To that end, we start as a reference from Case Inconsistency Level (CIL), which counts the number of inconsistent project pairs in a data set to evaluate its consistency. Based on a follow-up evaluation with a large sample set, we depict that CIL is not effective in evaluating the quality of certain data sets. By studying the problems associated with CIL and eliminating them, we propose an improved metric called Similar Case Inconsistency Level (SCIL). Our empirical evaluation with 54 data samples derived from six large project data sets shows that SCIL can distinguish between consistent and inconsistent data sets, and that prediction models for software development effort and productivity built from consistent data sets achieve indeed a relatively higher accuracy.