Understanding user understanding: determining correctness of generated program invariants
Understanding user understanding: determining correctness of generated program invariants
复制标题
了解用户理解：确定生成的程序不变量的正确性
DOI：
--
复制
发表时间：
2012
期刊：
影响因子：
--
通讯作者：
G. Rothermel
中科院分区：
文献类型：
--
作者：
Matthew Staats;Shin Hong;Moonzoo Kim;G. Rothermel
最近，自动生成测试预言机的工作已经开始，这是完全自动化测试过程所必需的。这种自动化的一种方法涉及从程序执行中提取不变量的动态不变量生成。然而，要使用这种不变量作为测试预言机，有必要区分正确的不变量和不正确的不变量，这是一个目前需要人工干预的过程。在这项工作中，我们研究这个过程。特别是，我们研究了30个用户的能力，在两个实证研究，从三个Java程序生成的不变量进行分类。我们的研究结果表明，用户很难对生成的不变量进行分类：平均而言，他们错误分类了9.1%至31.7%的正确不变量和26.1%至58.6%的不正确不变量。这些结果与以前的研究，表明用户分类是很容易的，并表明，需要做进一步的工作，以弥合理论上的动态不变量生成的有效性之间的差距差距，用户在实践中应用它的能力。沿着这些思路，我们提出了今后工作的几个领域。
Recently, work has begun on automating the generation of test oracles, which are necessary to fully automate the testing process. One approach to such automation involves dynamic invariant generation which extracts invariants from program executions. To use such invariants as test oracles, however, it is necessary to distinguish correct from incorrect invariants, a process that currently requires human intervention. In this work we examine this process. In particular, we examine the ability of 30 users, across two empirical studies, to classify invariants generated from three Java programs. Our results indicate that users struggle to classify generated invariants: on average, they misclassify 9.1% to 31.7% of correct invariants and 26.1%-58.6% of incorrect invariants. These results contradict prior studies that suggest that classification by users is easy, and indicate that further work needs to be done to bridge the gap between the effectiveness of dynamic invariant generation in theory, and the ability of users to apply it in practice. Along these lines, we suggest several areas for future work.