A similarity-based approach for test case prioritization using historical failure data
A similarity-based approach for test case prioritization using historical failure data
复制标题
DOI：
10.1109/issre.2015.7381799
复制
发表时间：
2015-11
期刊：
影响因子：
--
通讯作者：
Tanzeem Bin Noor;H. Hemmati
中科院分区：
文献类型：
--
作者：
Tanzeem Bin Noor;H. Hemmati
测试用例优先级排序是软件质量保证的一个关键因素，特别是在回归测试中。通常情况下，测试用例的优先级是以它们更早检测到潜在故障的方式确定的。测试用例的有效性，在故障检测方面，估计使用质量指标，如代码覆盖率，大小，和历史故障检测。先前的研究表明，以前失败的测试用例很可能在下一个版本中再次失败，因此，它们在优先级排序时排名很高。然而，在实践中，失败的测试用例可能与先前失败的测试用例不完全相同，但非常相似，例如，当新的失败测试是旧的失败测试的稍微修改的版本以捕获未检测到的故障时。在本文中，我们定义了一类度量，估计测试用例的质量使用它们的相似性，以前失败的测试用例。我们已经进行了几个实验与五个真实的世界的开源软件系统，与真实的故障，以评估这些质量指标的有效性。我们的研究结果表明，我们提出的基于相似性的质量措施是显着更有效的测试用例的优先级相比，现有的测试用例的质量措施。
Test case prioritization is a crucial element in software quality assurance in practice, specially, in the context of regression testing. Typically, test cases are prioritized in a way that they detect the potential faults earlier. The effectiveness of test cases, in terms of fault detection, is estimated using quality metrics, such as code coverage, size, and historical fault detection. Prior studies have shown that previously failing test cases are highly likely to fail again in the next releases, therefore, they are highly ranked, while prioritizing. However, in practice, a failing test case may not be exactly the same as a previously failed test case, but quite similar, e.g., when the new failing test is a slightly modified version of an old failing one to catch an undetected fault. In this paper, we define a class of metrics that estimate the test cases quality using their similarity to the previously failing test cases. We have conducted several experiments with five real world open source software systems, with real faults, to evaluate the effectiveness of these quality metrics. The results of our study show that our proposed similarity-based quality measure is significantly more effective for prioritizing test cases compared to existing test case quality measures.