Creating Equitable Grading Practices with Rubrics: A Teaching Assistant Training Activity
Creating Equitable Grading Practices with Rubrics: A Teaching Assistant Training Activity
复制标题
使用评分标准创建公平的评分实践：助教培训活动
DOI：
10.1145/3568812.3603485
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
N. Roy
中科院分区：
文献类型：
--
作者：
Rodrigo Borela;N. Roy
在大型计算机科学（CS）课程中，手动评分编码作业是一项具有挑战性的后勤任务。对代码正确性的评估是主观的，会导致分级偏差和不一致。当多个助教（助教）对同一问题的不同提交进行评分时，这个问题会更加严重。针对这些挑战的一个潜在解决方案是使用评分标准，它通过采用学生作业质量的特定标准提供了一种结构化的评分方法[1,2]。他们可以帮助减轻教师的内隐偏见，并尽量减少评分差异，即使涉及多个评分者[2,3]。此外，标题可以激励学生改进他们的工作[4]，加强他们在CS课程[4]中的学习，并为多媒体项目[5]提供客观的评价。然而，有效性、可靠性和公平性问题仍然适用于它们的使用。考虑到编码的主观性，根据概念理解而不仅仅是最终输出来分配部分分数，可以促进公平评分，因为最终输出可能并不总是反映学生对主题的掌握程度。因此，规则的设计必须简单、具体，并且易于教师、评分人员和学生理解。在这项研究中，我们进行了一次实践助教培训研讨会，以探索在计算机科学本科核心课程中，评分标准对编码作业的有效性。讲习班向与会者提供了标准和评分员培训，以评估标准对评分偏差和一致性的影响。此外，研讨会介绍了基于学生概念理解的部分分的使用，旨在促进公平和客观的评分，同时也解决了这种方法的局限性。工作坊是为不同组别的助教而设，共50人参加。首先，参与者给出了两个错误的问题解决方案，并要求他们根据自己的判断给它们打分。在第二轮中，他们用一个分数给不同的解决方案打分。两轮评分的结果匿名报告，并实时图表显示分数分布。统计分析显示，两组学生的成绩差异显著减少，从1到10，分别从5.55到0.66和从4.28到1.11。这种定量和可视化的分析有助于助教理解规则的重要性，并使他们能够讨论开发和部署规则的偏差和最佳实践。助教们的非正式反馈表明，讲习班有效地展示了制定良好规则的重要性。工作坊的设计扩展了之前其他学科的工作坊，展示了计算机科学中评分的独特方面，例如解决编码问题的不同方法和引入可视化来理解评分差异。这个工作坊展示了如何运用标准和定量数据可视化分析来促进公平的评分做法，并强调道德问题在计算机科学教育培训中的重要性。受数学、化学和生物学中使用规则的研讨会的启发，我们通过适应编码中固有的各种解决问题的途径进行了创新，从而强调概念上的分级而不是过程上的分级。我们方法的独特之处在于集成了数据可视化组件，以帮助理解分级差异。这个工作坊的设计可以扩展到各种教学主题，允许对小规模实验进行实时分析，从而扩大其适用性并塑造未来的教学策略。
Manually grading coding assignments in large computer science (CS) classes is a challenging logistical task. The evaluation of code correctness is subjective, leading to grading bias and inconsistencies. This problem is exacerbated when multiple teaching assistants (TAs) grade different submissions of the same problem. One potential solution to these challenges is the use of rubrics, which offer a structured approach to grading by adopting specific criteria for the quality of student work [1, 2]. They can help mitigate instructors' implicit biases and minimize grading variance even when multiple raters are involved [2, 3]. Moreover, rubrics can motivate students to improve their work [4], enhance their learning in CS courses [4], and provide an objective evaluation of multimedia projects [5]. However, validity, reliability, and fairness issues still apply to their use [6]. To account for the subjective nature of coding, assigning partial points based on conceptual understanding rather than just the final output promotes equitable grading since the final output may not always reflect the student's grasp of the topics [6]. Therefore, rubrics must be designed to be simple, specific, and easy to understand for instructors, graders, and students. In this study, we conducted a hands-on TA training workshop to explore the effectiveness of rubrics in grading coding assignments in a core undergraduate CS course. The workshop provided participants with rubrics and rater training to assess the impact of rubrics on grading bias and consistency. Additionally, the workshop introduced the use of partial points based on students' conceptual understanding, aiming to promote fair and objective grading while also addressing the limitations of this approach. The workshop was conducted for distinct groups of TAs, totaling 50 participants. Firstly, participants were given two sample incorrect solutions to a problem and asked to grade them at their own discretion. In the second round, they re-graded the solutions using a rubric with assigned points to different solution components. The results of the two grading rounds were reported anonymously, and a real-time graph displayed the grade distribution. Statistical analysis revealed a significant reduction in grade variance for both student groups, from 5.55 to 0.66 and from 4.28 to 1.11, respectively, on a scale from 1 to 10. This quantitative and visual analysis helped TAs understand the importance of rubrics and enabled them to discuss biases and best practices for developing and deploying rubrics. TAs' informal feedback indicated the workshop's effectiveness in demonstrating the relevance of creating good rubrics. The workshop design expands on previous workshops for other disciplines by showcasing the unique aspects of grading in computer science, such as the different approaches to solving coding problems and the introduction of visualizations to understand grading variances. This workshop showcases the application of rubrics and quantitative data visual analytics to promote equitable grading practices and underscore the significance of ethical issues in CS education training. Inspired by rubric-utilizing workshops in math, chemistry, and biology, we have innovated by accommodating the varied problem-solving paths inherent in coding, thus emphasizing conceptual over procedural grading. Unique to our approach is the integration of a data visualization component to help understand grading variance. This workshop's design can be extended to various pedagogical topics, allowing real-time analysis of small-scale experiments, thus broadening its applicability and shaping future teaching strategies.