Toward practical mutation analysis for evaluating the quality of student-written software tests
Toward practical mutation analysis for evaluating the quality of student-written software tests
复制标题
用于评估学生编写的软件测试质量的实用突变分析
DOI：
10.1145/2493394.2493402
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
S. Edwards
中科院分区：
文献类型：
--
作者：
Z. Shams;S. Edwards
软件测试正在添加到许多学校的编程课程中，但是当前用于评估学生编写测试的评估技术并不完美。代码覆盖措施通常在实践中使用，但是它们有局限性，有时高估了测试的真实质量。其他人则建议使用突变分析，但是突变分析对课堂使用构成了许多实际障碍。本文介绍了一种对学生编写测试的突变分析的新方法，该方法在教育使用方面更为实用，尤其是在自动化的分级环境中。这种方法结合了几种技术来生成一种新的解决方案，以解决更传统的突变分析所带来的缺点。在CS1和CS2课程的背景下对这种方法的评估说明了它与代码覆盖分析的不同。然而，与此同时，评估结果还引发了CS教育工作者有关测试质量更全面评估的相对价值的关注问题，这些质量更全面评估的价值，为学生提供重要设计自由的更开放式任务，提供的成本高质量的参考解决方案，以支持更好的质量评估，以及需要更深入测试的支撑任务的成本，例如GUI分配。
Software testing is being added to programming courses at many schools, but current assessment techniques for evaluating student-written tests are imperfect. Code coverage measures are typically used in practice, but they have limitations and sometimes overestimate the true quality of tests. Others have proposed using mutation analysis instead, but mutation analysis poses a number of practical obstacles to classroom use. This paper describes a new approach to mutation analysis of student-written tests that is more practical for educational use, especially in an automated grading context. This approach combines several techniques to produce a novel solution that addresses the shortcomings raised by more traditional mutation analysis. An evaluation of this approach in the context of both CS1 and CS2 courses illustrates how it differs from code coverage analysis. At the same time, however, the evaluation results also raise questions of concern for CS educators regarding the relative value of more comprehensive assessment of test quality, the value of more open-ended assignments that offer significant design freedom for students, the cost of providing higher-quality reference solutions in order to support better quality assessment, and the cost of supporting assignments that require more intensive testing, such as GUI assignments.