Exploring Differences Between Instructors' Exams and How These Differences Produce Scores that Could Inaccurately and Inequitably Represent Student Understanding

在许多STEM课程中，学生的考试成绩决定了他们的课程成绩。 反过来，当平均在一起时，课程成绩决定了每个学生的平均成绩，这可能会影响他们对STEM专业的坚持以及他们进入专业或研究生院的竞争力。因此，重要的是，这些考试准确和公平地衡量学生对他们应该测试的主题的理解。很少有研究来确定特定的考试问题是否准确地衡量学生的理解，或者它们是否对所有学生都公平。 亚利桑那州立大学和华盛顿大学之间的这个合作项目将试图通过分析不同导师教授的入门生物课程考试的问题来填补这一知识空白。 他们将检查不同类型的问题之间的关系，学生对问题的分数，以及学生对问题应该测试的概念的理解。 这些信息有可能帮助生物教师更公平，更准确地测试学生对生物学的理解。 它还可以为构建与其他STEM学科相关的公平、准确的考试问题提供指导。该项目有四个关键目标：（1）描述同一生物学入门课程各部分教师生成的生物学考试的组成;（2）描述导致学生在问题上表现不同的问题元素;（3）确定修改问题是否可以减少学生表现的差异;（4）将考试成绩与学生对生物学的概念理解相关联。该项目将侧重于同一区域网络内多个机构提供的教授相同生物学入门课程的不同教师的考试。没有发表的研究探讨了在生物学导论课程中自动生成的考试中问题表现的差异。此外，该项目将是迄今为止对任何STEM学科中自动生成的考试和问题进行的最大，最全面的分析，从而深入了解哪些因素可能影响单个问题的表现差异。 该项目可以帮助教师和研究人员更加了解导致对某些学生群体的不公平评价的问题的要素，从而更准确和公平地衡量学生的理解。

In many STEM courses, students' exam scores determine their course grades. In turn, when averaged together, course grades determine each student's grade point average, which can affect their persistence in a STEM major and their competitiveness for admission to professional or graduate schools. Thus, it is important that these exams accurately and equitably measure students' understanding of the subject matter that they are supposed to test. Little research has been done to determine whether specific exam questions accurately measure student understanding or whether they are fair to all students. This collaborative project between Arizona State University and University of Washington will try to fill this gap in knowledge by analyzing questions on introductory biology course exams taught by different instructors. They will examine the relationships between different types of questions, student scores on the questions, and student understanding of the concept that the question is supposed to test. This information has the potential to help biology instructors more fairly and accurately test student understanding of biology. It may also provide guidelines for building fair and accurate exam questions that are relevant to other STEM disciplines.This project has four key goals: (1) characterizing the composition of instructor-generated biology exams across sections of the same introductory biology course; (2) characterizing elements of questions that result in students performing differently on questions; (3) determining if modifying questions can diminish differences in student performance; and (4) correlating exam scores to students' conceptual understanding of biology. This project will focus on the exams of different instructors teaching the same introductory biology course offered across multiple institutions within the same regional network. No published studies have explored differences in question performance on instructor-generated exams in introductory biology courses. Further, this project would be the largest, most comprehensive analysis of instructor-generated exams and questions in any STEM discipline done so far, providing insights into what factors may affect performance differences on individual questions. This project can help instructors and researchers become more aware of elements of questions that result in unfair evaluation of certain groups of students, leading to more accurate and equitable measurement of student understanding.