Who Tests the Testers? Avoiding the Perils of Automated Testing
Who Tests the Testers? Avoiding the Perils of Automated Testing
复制标题
谁来测试测试人员？
DOI：
10.1145/3230977.3230999
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Fisler, Kathi
中科院分区：
文献类型：
--
作者：
Wrenn, John;Krishnamurthi, Shriram;Fisler, Kathi
教师经常使用自动评估方法来评估学生实现的语义质量，有时还评估测试套件的语义质量。在这项工作中，我们提炼出文献中的各种自动化评估方法，归结为一对评估模型。我们确定每个模型中的病理评估结果，指出潜在的方法缺陷。这些理论缺陷广泛地威胁着这些技术的有效性，我们实际上在一门编程入门课程的多个作业中观察到了它们。我们建议进行调整，以弥补这些缺陷，然后在这些相同的任务中证明，我们的干预措施提高了评估的准确性。我们相信，通过这些调整，教员可以大大提高自动化评估的准确性。
Instructors routinely use automated assessment methods to evaluate the semantic qualities of student implementations and, sometimes, test suites. In this work, we distill a variety of automated assessment methods in the literature down to a pair of assessment models. We identify pathological assessment outcomes in each model that point to underlying methodological flaws. These theoretical flaws broadly threaten the validity of the techniques, and we actually observe them in multiple assignments of an introductory programming course. We propose adjustments that remedy these flaws and then demonstrate, on these same assignments, that our interventions improve the accuracy of assessment. We believe that with these adjustments, instructors can greatly improve the accuracy of automated assessment.