Automated Test Data Generation for Coverage: Haven't We Solved This Problem Yet?
Automated Test Data Generation for Coverage: Haven't We Solved This Problem Yet?
复制标题
DOI：
10.1109/taicpart.2009.15
复制
发表时间：
2009-09
期刊：
影响因子：
--
通讯作者：
Kiran Lakhotia;Phil McMinn;Mark Harman
中科院分区：
文献类型：
--
作者：
Kiran Lakhotia;Phil McMinn;Mark Harman
虽然有很多证据表明基于混合和搜索的测试都可以优于随机测试，但很少有工作证明这两种技术在完整的现实世界软件应用程序中的有效性。因此，许多研究人员不仅怀疑这两种方法的可扩展性，而且怀疑它们对生产代码的适用性。本文对四个大型开源应用程序的源代码应用了 concolic 工具 CUTE 和基于搜索的工具 AUSTIN 进行了实证研究。每个工具都是“开箱即用”的；这不需要编写额外的代码来特殊处理任何单个主题，也不需要调整工具的参数。也许令人惊讶的是，结果表明这两种工具最多只能获得适度的代码覆盖率。改进自动化测试数据生成器以实现更高水平的代码覆盖率仍然存在一些挑战。
Whilst there is much evidence that both concolic and search based testing can outperform random testing, there has been little work demonstrating the effectiveness of either technique with complete real world software applications. As a consequence, many researchers have doubts not only about the scalability of both approaches but also their applicability to production code. This paper performs an empirical study applying a concolic tool, CUTE, and a search based tool, AUSTIN, to the source code of four large open source applications. Each tool is applied `out of the box'; that is without writing additional code for special handling of any of the individual subjects, or by tuning the tools' parameters. Perhaps surprisingly, the results show that both tools can only obtain at best a modest level of code coverage. Several challenges remain for improving automated test data generators in order to achieve higher levels of code coverage.