Do student programmers all tend to write the same software tests?
Do student programmers all tend to write the same software tests?
复制标题
学生程序员都倾向于编写相同的软件测试吗？
DOI：
10.1145/2591708.2591757
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
Z. Shams
中科院分区：
文献类型：
--
作者：
S. Edwards;Z. Shams
尽管许多教育工作者将软件测试实践添加到其编程任务中，但使用陈述覆盖范围或分支机构覆盖的学生编写的测试的有效性是局限性的，而研究人员已经开始调查评估学生写入的测试的替代方法在真实的人为缺陷的数量方面，这些测试的质量可以检测到一个实验使用为CS2数据结构编写的101个程序进行，学生使用基于数组的和基于链接的表示，以两种方式实施队列。使用先前工作的技术，我们能够近似学生解决方案集合中存在的错误数量，并确定每个学生写入的哪个错误测试套件。结果表明，尽管他们的解决方案的平均分支机构为95.4％，但他们的测试套件只能检测到整个程序人群中的平均分支机构90％的学生测试套件的相似程度很高。行为而不是撰写旨在检测隐藏错误的测试。
While many educators have added software testing practices to their programming assignments, assessing the effectiveness of student-written tests using statement coverage or branch coverage has limitations. While researchers have begun investigating alternative approaches to assessing student-written tests, this paper reports on an investigation of the quality of student written tests in terms of the number of authentic, human-written defects those tests can detect. An experiment was conducted using 101 programs written for a CS2 data structures assignment where students implemented a queue two ways, using both an array-based and a link-based representation. Students were required to write their own software tests and graded in part on the branch coverage they achieved. Using techniques from prior work, we were able to approximate the number of bugs present in the collection of student solutions, and identify which of these were detected by each student-written test suite. The results indicate that, while students achieved an average branch coverage of 95.4% on their own solutions, their test suites were only able to detect an average of 13.6% of the faults present in the entire program population. Further, there was a high degree of similarity among 90% of the student test suites. Analysis of the suites suggest that students were following naïve, "happy path" testing, writing basic test cases covering mainstream expected behavior rather than writing tests designed to detect hidden bugs. These results suggest that educators should strive to reinforce test design techniques intended to find bugs, rather than simply confirming that features work as expected.