How does combinatorial testing perform in the real world: an empirical study
How does combinatorial testing perform in the real world: an empirical study
复制标题
DOI：
10.1007/s10664-019-09799-2
复制
发表时间：
2020-04
影响因子：
4.1
通讯作者：
Linghuan Hu;W. Wong;D. Kuhn;R. Kacker
中科院分区：
文献类型：
--
作者：
Linghuan Hu;W. Wong;D. Kuhn;R. Kacker
研究表明，组合测试（CT）可以有效地检测软件系统中的故障。通过关注系统中不同因素之间的相互作用，CT显示出其检测故障的潜力，特别是那些只能通过多个因素的值的特定组合（多因素故障）来揭示的故障。然而，CT是否足够实用，可以在行业中应用？它能比其他行业青睐的技术更有效吗？在实践中应用CT时是否存在挑战？这些研究问题仍然在工业环境的背景下。在本文中，我们提出了一个实证研究的CT五个工业系统的真实的故障。详细的输入空间模型（ISM）的建设，如因素识别和赋值，包括。我们将CT检测到的故障与内部测试团队使用其他方法检测到的故障进行了比较，结果表明，尽管存在一些挑战，但CT是检测工业环境中软件系统真实的故障，特别是多因素故障的有效技术。提供了观察结果和经验教训，以进一步提高故障检测的有效性，克服各种挑战。
Studies have shown that combinatorial testing (CT) can be effective for detecting faults in software systems. By focusing on the interactions between different factors of a system, CT shows its potential for detecting faults, especially those that can be revealed only by the specific combinations of values of multiple factors (multi-factor faults). However, is CT practical enough to be applied in the industry? Can it be more effective than other industry-favored techniques? Are there any challenges when applying CT in practice? These research questions remain in the context of industrial settings. In this paper, we present an empirical study of CT on five industrial systems with real faults. The details of the input space model (ISM) construction, such as factor identification and value assignment, are included. We compared the faults detected by CT with those detected by the in-house testing teams using other methods, and the results suggest that despite some challenges, CT is an effective technique to detect real faults, especially multi-factor faults, of software systems in industrial settings. Observations and lessons learned are provided to further improve the fault detection effectiveness and overcome various challenges.