Optimizing for the Number of Tests Generated in Search Based Test Data Generation with an Application to the Oracle Cost Problem
Optimizing for the Number of Tests Generated in Search Based Test Data Generation with an Application to the Oracle Cost Problem
复制标题
DOI：
10.1109/icstw.2010.31
复制
发表时间：
2010-04
期刊：
影响因子：
--
通讯作者：
M. Harman;Sung Gon Kim;Kiran Lakhotia;Phil McMinn;S. Yoo
中科院分区：
文献类型：
--
作者：
M. Harman;Sung Gon Kim;Kiran Lakhotia;Phil McMinn;S. Yoo
以往基于搜索的测试数据生成方法往往侧重于覆盖范围，而不是甲骨文成本。虽然人们可能希望系统应该有模型、可检查的规范和/或合同驱动的开发，但遗憾的是，这仍然只是一种愿望；在许多实际情况下，系统行为必须由人工进行检查。这种艰苦的检查过程会产生巨大的成本--oracle 成本，而以往的自动测试数据生成工作往往会忽略这一点。降低甲骨文成本的一个简单方法是减少生成测试的数量。在本文中，我们介绍了三种算法，它们可以在不影响覆盖率的情况下做到这一点。我们介绍了这三种算法在五个基准程序中的有效性实证研究结果，这五个基准程序包含非琐碎的分支覆盖搜索空间。结果表明，在不损失覆盖率的情况下，通过基于搜索的测试减少测试用例的数量确实是可能的。
Previous approaches to search based test data generation tend to focus on coverage, rather than oracle cost. While there may be an aspiration that systems should have models, checkable specifications and/or contract driven development, this sadly remains an aspiration; in many real cases, system behaviour must be checked by a human. This painstaking checking process forms a significant cost, the oracle cost, which previous work on automated test data generation tends to overlook. One simple way to reduce oracle cost consists of reducing the number of tests generated. In this paper we introduce three algorithms which do this without compromising coverage achieved. We present the results of an empirical study of the effectiveness of the three algorithms on five benchmark programs containing non trivial search spaces for branch coverage. The results indicate that it is, indeed, possible to make reductions in the number of test cases produced by search based testing, without loss of coverage.