Evaluating non-adequate test-case reduction
Evaluating non-adequate test-case reduction
复制标题
DOI：
10.1145/2970276.2970361
复制
发表时间：
2016-08
期刊：
影响因子：
--
通讯作者：
Mohammad Amin Alipour;A. Shi;Rahul Gopinath;D. Marinov;Alex Groce
中科院分区：
文献类型：
--
作者：
Mohammad Amin Alipour;A. Shi;Rahul Gopinath;D. Marinov;Alex Groce
给定两个测试用例，一个较大，一个较小，较小的测试用例在许多方面都是首选的。较小的测试用例通常运行得更快，更容易理解，也更便于调试。然而，与较大的测试用例相比，较小的测试用例往往覆盖更少的代码和检测到更少的错误。传统的研究侧重于在保留代码覆盖率的同时减少测试用例，而最近的工作引入了在保留代码覆盖率的同时减少个别测试用例而不是测试套件的想法。其他最近的工作建议，通过甚至不保留所有代码覆盖率来不充分地减少测试套件。本文实证地评估了这两种思想的新组合--测试用例的非充分约简，它允许在测试用例大小和错误检测之间进行广泛的权衡。我们的研究引入并评估了C%覆盖率减少(测试用例被减少到至少保留其原始覆盖率的C%)和N突变减少(测试用例被减少到至少N个它最初杀死的突变体)。我们评估了四个实际C项目在不同C%和N值下的缩减权衡：Mozilla的SpiderMonkey JavaScript引擎、YAFFS2闪存文件系统、grep和Gzip。结果表明，在保留大量测试用例的故障检测能力的同时，大大减少测试用例的大小是可能的。
Given two test cases, one larger and one smaller, the smaller test case is preferred for many purposes. A smaller test case usually runs faster, is easier to understand, and is more convenient for debugging. However, smaller test cases also tend to cover less code and detect fewer faults than larger test cases. Whereas traditional research focused on reducing test suites while preserving code coverage, recent work has introduced the idea of reducing individual test cases, rather than test suites, while still preserving code coverage. Other recent work has proposed non-adequately reducing test suites by not even preserving all the code coverage. This paper empirically evaluates a new combination of these two ideas, non-adequate reduction of test cases, which allows for a wide range of trade-offs between test case size and fault detection. Our study introduces and evaluates C%-coverage reduction (where a test case is reduced to retain at least C% of its original coverage) and N-mutant reduction (where a test case is reduced to kill at least N of the mutants it originally killed). We evaluate the reduction trade-offs with varying values of C% and N for four real-world C projects: Mozilla's SpiderMonkey JavaScript engine, the YAFFS2 flash file system, Grep, and Gzip. The results show that it is possible to greatly reduce the size of many test cases while still preserving much of their fault-detection capability.