Software testing with code-based test generators: data and lessons learned from a case study with an industrial software component
Software testing with code-based test generators: data and lessons learned from a case study with an industrial software component
复制标题
使用基于代码的测试生成器进行软件测试：从工业软件组件案例研究中吸取的数据和经验教训
DOI：
--
复制
发表时间：
2013
影响因子：
1.9
通讯作者：
Ali Muhammad
中科院分区：
文献类型：
--
作者：
Pietro Braione;G. Denaro;Andrea Mattavelli;Mattia Vivanti;Ali Muhammad
自动生成有效的测试套件通过在合理的努力和成本范围内推广经过广泛测试的软件，有望对测试实践产生重大影响。基于代码的测试生成器依靠被测软件的源代码来识别测试目标并相应地引导测试用例生成过程。目前，该主题最成熟的提案来自随机测试、动态符号执行和基于搜索的测试的研究。本文研究了一组最新研究测试生成器对一系列具有重要领域特定特性的工业程序的有效性。这些程序是实时和安全关键控制系统的软件组件的一部分，并集成在 LabVIEW（一种用于设计嵌入式系统的图形语言）中指定的控制任务中。这项研究的结果增强了关于测试生成器的优点和缺点的可用知识体系。经验数据表明，测试生成器可以真正暴露主题软件中微妙的（以前未知的）错误，并且以互补甚至协同的方式使用不同类型的测试生成方法可能是有价值的。此外，我们的实验指出，对浮点算术和非线性计算的支持是充分发挥工业中基于符号执行的原型潜力的一个重要里程碑。
Automatically generating effective test suites promises a significant impact on testing practice by promoting extensively tested software within reasonable effort and cost bounds. Code-based test generators rely on the source code of the software under test to identify test objectives and to steer the test case generation process accordingly. Currently, the most mature proposals on this topic come from the research on random testing, dynamic symbolic execution, and search-based testing. This paper studies the effectiveness of a set of state-of-the-research test generators on a family of industrial programs with nontrivial domain-specific peculiarities. These programs are part of a software component of a real-time and safety-critical control system and integrate in a control task specified in LabVIEW, a graphical language for designing embedded systems. The result of this study enhances the available body of knowledge on the strengths and weaknesses of test generators. The empirical data indicate that the test generators can truly expose subtle (previously unknown) bugs in the subject software and that there can be merit in using different types of test generation approaches in a complementary, even synergic fashion. Furthermore, our experiment pinpoints the support for floating point arithmetics and nonlinear computations as a major milestone in the path to exploiting the full potential of the prototypes based on symbolic execution in industry.