Exploring Differences Between Instructors' Exams and How These Differences Produce Scores that Could Inaccurately and Inequitably Represent Student Understanding

在许多STEM课程中，学生的考试成绩决定了他们的课程成绩。反过来，当平均起来，课程成绩决定了每个学生的平均绩点，这可能会影响他们在STEM专业的坚持，以及他们进入专业或研究生院的竞争力。因此，这些考试准确而公平地衡量学生对他们应该测试的主题的理解是很重要的。很少有人做过研究来确定具体的考试问题是否能准确地衡量学生的理解程度，或者它们是否对所有学生都公平。亚利桑那州立大学和华盛顿大学之间的这个合作项目将试图通过分析由不同老师教授的生物学入门课程考试中的问题来填补这一知识空白。他们将检查不同类型的问题之间的关系，学生在问题上的得分，以及学生对问题应该测试的概念的理解。这些信息有可能帮助生物学教师更公平、更准确地测试学生对生物学的理解。它还可以为构建与其他STEM学科相关的公平和准确的考试问题提供指导。该项目有四个主要目标：(1)描述同一门生物入门课程中不同部分的教师生成的生物考试的组成特征；(2)描述导致学生在问题上表现不同的问题要素；(3)确定修改问题是否可以减少学生成绩的差异；(4)考试成绩与学生对生物学概念理解的关系。该项目将重点关注在同一区域网络内的多个机构中教授同一门生物学入门课程的不同讲师的考试。没有发表的研究探讨了在生物学入门课程中教师自编考试中问题表现的差异。此外，该项目将是迄今为止在任何STEM学科中对教师生成的考试和问题进行的最大，最全面的分析，从而深入了解哪些因素可能影响个别问题的表现差异。这个项目可以帮助教师和研究人员更加了解导致对某些学生群体进行不公平评估的问题因素，从而更准确、更公平地衡量学生的理解程度。

In many STEM courses, students' exam scores determine their course grades. In turn, when averaged together, course grades determine each student's grade point average, which can affect their persistence in a STEM major and their competitiveness for admission to professional or graduate schools. Thus, it is important that these exams accurately and equitably measure students' understanding of the subject matter that they are supposed to test. Little research has been done to determine whether specific exam questions accurately measure student understanding or whether they are fair to all students. This collaborative project between Arizona State University and University of Washington will try to fill this gap in knowledge by analyzing questions on introductory biology course exams taught by different instructors. They will examine the relationships between different types of questions, student scores on the questions, and student understanding of the concept that the question is supposed to test. This information has the potential to help biology instructors more fairly and accurately test student understanding of biology. It may also provide guidelines for building fair and accurate exam questions that are relevant to other STEM disciplines.This project has four key goals: (1) characterizing the composition of instructor-generated biology exams across sections of the same introductory biology course; (2) characterizing elements of questions that result in students performing differently on questions; (3) determining if modifying questions can diminish differences in student performance; and (4) correlating exam scores to students' conceptual understanding of biology. This project will focus on the exams of different instructors teaching the same introductory biology course offered across multiple institutions within the same regional network. No published studies have explored differences in question performance on instructor-generated exams in introductory biology courses. Further, this project would be the largest, most comprehensive analysis of instructor-generated exams and questions in any STEM discipline done so far, providing insights into what factors may affect performance differences on individual questions. This project can help instructors and researchers become more aware of elements of questions that result in unfair evaluation of certain groups of students, leading to more accurate and equitable measurement of student understanding.