Exploring Differences Between Instructors' Exams and How These Differences Produce Scores that Could Inaccurately and Inequitably Represent Student Understanding

在许多STEM课程中，学生的考试成绩决定了他们的课程成绩。反过来，当课程成绩加在一起时，决定了每个学生的平均成绩，这可能会影响他们在STEM专业的毅力，以及他们进入专业学校或研究生院的竞争力。因此，这些考试准确而公平地衡量学生对他们应该测试的主题的理解是很重要的。几乎没有进行研究来确定具体的考题是否准确地衡量了学生的理解能力，或者它们是否对所有学生公平。亚利桑那州立大学和华盛顿大学的这个合作项目将试图通过分析由不同教师教授的生物入门课程考试的问题来填补这一知识空白。他们将研究不同类型的问题之间的关系，学生在问题上的分数，以及学生对问题应该测试的概念的理解。这些信息有可能帮助生物教师更公平、更准确地测试学生对生物的理解。这个项目有四个关键目标：(1)描述同一门生物入门课程中由教师编写的生物考试的组成；(2)描述导致学生在问题上表现不同的问题元素；(3)确定修改问题是否可以缩小学生表现的差异；以及(4)将考试分数与学生对生物的概念理解联系起来。本项目将侧重于不同讲师教授同一门生物学入门课程的考试，这些课程由同一地区网络内的多所院校提供。目前还没有公开发表的研究探讨教师在生物入门课程考试中的问题表现差异。此外，该项目将是迄今为止所有STEM学科中对教师生成的考试和问题进行的最大、最全面的分析，为了解哪些因素可能影响个别问题的表现提供了洞察力。这个项目可以帮助教师和研究人员更多地了解导致对某些学生群体进行不公平评估的问题的要素，从而更准确和公平地衡量学生的理解。

In many STEM courses, students' exam scores determine their course grades. In turn, when averaged together, course grades determine each student's grade point average, which can affect their persistence in a STEM major and their competitiveness for admission to professional or graduate schools. Thus, it is important that these exams accurately and equitably measure students' understanding of the subject matter that they are supposed to test. Little research has been done to determine whether specific exam questions accurately measure student understanding or whether they are fair to all students. This collaborative project between Arizona State University and University of Washington will try to fill this gap in knowledge by analyzing questions on introductory biology course exams taught by different instructors. They will examine the relationships between different types of questions, student scores on the questions, and student understanding of the concept that the question is supposed to test. This information has the potential to help biology instructors more fairly and accurately test student understanding of biology. It may also provide guidelines for building fair and accurate exam questions that are relevant to other STEM disciplines.This project has four key goals: (1) characterizing the composition of instructor-generated biology exams across sections of the same introductory biology course; (2) characterizing elements of questions that result in students performing differently on questions; (3) determining if modifying questions can diminish differences in student performance; and (4) correlating exam scores to students' conceptual understanding of biology. This project will focus on the exams of different instructors teaching the same introductory biology course offered across multiple institutions within the same regional network. No published studies have explored differences in question performance on instructor-generated exams in introductory biology courses. Further, this project would be the largest, most comprehensive analysis of instructor-generated exams and questions in any STEM discipline done so far, providing insights into what factors may affect performance differences on individual questions. This project can help instructors and researchers become more aware of elements of questions that result in unfair evaluation of certain groups of students, leading to more accurate and equitable measurement of student understanding.