Metamorphic Testing with Causal Graphs
Metamorphic Testing with Causal Graphs
复制标题
DOI：
10.1109/icst57152.2023.00023
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Andrew G. Clark;Michael Foster;Neil Walkinshaw;R. Hierons
中科院分区：
文献类型：
--
作者：
Andrew G. Clark;Michael Foster;Neil Walkinshaw;R. Hierons
变形测试提供了一种生成适用于大型输入空间的简洁测试预言的方法。为此，它依赖于变质关系的表述，这通常需要广泛的领域专业知识和人力投入。为了解决这个问题，我们提出了一种基于模型的测试方法，它可以自动生成变质关系和相关测试。我们的方法是基于这样一种观察，即变质测试从根本上来说是一项因果任务。我们展示了如何利用因果推理领域的轻量级基于图的建模技术来指定被测系统的因果属性。通过一系列对照实验，我们发现该方法对错误描述具有健壮性，并且当与适当的测试生成策略相结合时，可以测试回避因果关系(即那些难以测试和观察的因果关系)。我们还将该方法应用于Defects4J框架中的两个案例研究，其中包含影响因果行为的已知错误。这些案例研究的结果表明，该方法不仅有助于捕获影响因果结构的错误，而且还可以提醒用户规范中的不准确。
Metamorphic testing provides a means by which to generate succinct test oracles that can apply to large input spaces. For this it depends on the formulation of metamorphic relations, which generally require extensive domain expertise and human input. To address this problem, we present a model-based testing approach that can automatically generate metamorphic relations and associated tests. Our approach is motivated by the observation that metamorphic testing is a fundamentally causal task. We show how it is possible to leverage lightweight graph-based modelling techniques from the field of causal inference to specify causal properties of the system-under-test. Through a series of controlled experiments, we find that the proposed approach is robust to misspecification and can test evasive causal relationships (i.e. those that are difficult to exercise and observe) when combined with an appropriate test generation strategy. We also apply the approach to two case studies from the Defects4J framework with known bugs that affect causal behaviour. The results of these case studies suggest that the approach is not only useful for catching bugs affecting causal structure, but also alerting the user to inaccuracies in the specification.