AUSTIN: An open source tool for search based software testing of C programs
AUSTIN: An open source tool for search based software testing of C programs
复制标题
DOI：
10.1016/j.infsof.2012.03.009
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
Kiran Lakhotia;M. Harman;Hamilton Gross
中科院分区：
文献类型：
--
作者：
Kiran Lakhotia;M. Harman;Hamilton Gross
背景尽管有大量关于基于搜索的软件测试（SBST）的出版物，但公开可用的工具仍然很少。本文介绍了 AUSTIN，一种公开可用的 C 语言开源 SBST 工具。1本文是之前工作的扩展 [1]。它包括在 AUSTIN 中实现的新爬山算法，以及对 AUSTIN 测试数据生成算法实现的不同指针处理技术的有效性和效率的调查。 目的评估 AUSTIN 在开源系统上实现的不同搜索算法在实现分支覆盖方面的有效性和效率。此外，为了将 AUSTIN 与非公开的、最先进的进化测试框架 (ETF) 进行比较。方法首先，我们使用开源基准测试中的示例函数以及常见的数据结构实现来检查本文中介绍的指针输入的决策过程与生成随机内存图的更简单的替代方案相比，在有效性和效率方面是否有所不同。第二项实证研究提出了关于 AUSTIN 与 ETF 相比的有效性和效率的两个替代假设。这些假设使用配对 Wilcoxon 检验进行检验。结果和结论第一项研究强调了本文中描述的指针输入决策过程的一些实际问题。特别是，如果被测代码包含的保护语句不足以对指针强制实施约束，那么与生成随机内存图的方法相比，对指针输入使用约束求解器可能不是最优的。第二项研究中使用的程序不需要对指针输入进行任何约束求解，并且由来自三个嵌入式汽车软件模块的八个重要的真实 C 函数组成。对于这些功能，AUSTIN 与 ETF 相比具有竞争力，其中六个功能实现了相同或更高的分支机构覆盖率。此外，对于 AUSTIN 分支机构覆盖率等于或更高的功能，AUSTIN 比 ETF 更高效。
CONTEXTDespite the large number of publications on Search-Based Software Testing (SBST), there remain few publicly available tools. This paper introduces AUSTIN, a publicly available open source SBST tool for the C language.1The paper is an extension of previous work [1]. It includes a new hill climb algorithm implemented in AUSTIN and an investigation into the effectiveness and efficiency of different pointer handling techniques implemented by AUSTIN’s test data generation algorithms.OBJECTIVETo evaluate the different search algorithms implemented within AUSTIN on open source systems with respect to effectiveness and efficiency in achieving branch coverage. Further, to compare AUSTIN against a non-publicly available, state-of-the-art Evolutionary Testing Framework (ETF).METHODFirst, we use example functions from open source benchmarks as well as common data structure implementations to check if the decision procedure for pointer inputs, introduced in this paper, differs in terms of effectiveness and efficiency compared to a simpler alternative that generates random memory graphs. A second empirical study formulates two alternate hypotheses regarding the effectiveness and efficiency of AUSTIN compared to the ETF. These hypotheses are tested using a paired Wilcoxon test.RESULTS AND CONCLUSIONThe first study highlights some practical problems with the decision procedure for pointer inputs described in this paper. In particular, if the code under test contains insufficient guard statements to enforce constraints over pointers, then using a constraint solver for pointer inputs may be suboptimal compared to a method that generates random memory graphs. The programs used in the second study do not require any constraint solving for pointer inputs and consist of eight non-trivial, real-world C functions drawn from three embedded automotive software modules. For these functions, AUSTIN is competitive compared to the ETF, achieving an equal or higher branch coverage for six of the functions. In addition, for functions where AUSTIN’s branch coverage is equal or higher, AUSTIN is more efficient than the ETF.