Unbiased, reliable, and valid student evaluations can still be unfair
Unbiased, reliable, and valid student evaluations can still be unfair
复制标题
公正、可靠和有效的学生评估仍然可能是不公平的
DOI：
10.1080/02602938.2020.1724875
复制
发表时间：
2020
影响因子：
4.4
通讯作者：
Natalie Valdes
中科院分区：
文献类型：
--
作者：
J. Esarey;Natalie Valdes
摘要关于学生评教的学术争论主要集中在学生评教是否有效、可靠和公正。在这篇文章中，我们假设的最乐观的条件，由实证文献支持的设置。具体来说，我们假设，设置是适度相关的教学质量（学生学习和教学的最佳实践），高度可靠的，并没有系统地歧视任何不相关的基础上。我们使用计算模拟表明，在理想的情况下，即使是谨慎和明智的使用SET来评估教师可以产生不可接受的高错误率：（a）SET分数的巨大差异无法可靠地确定最好的教师在成对比较，（B）超过四分之一的教师与评价或低于第20百分位数的教学质量高于中位数。这些问题是由于不精确的设置和教师质量之间的关系，即使它们是适度相关的存在。我们的模拟表明，评估指令使用多个不完美的措施，包括但不限于SET，可以产生一个更公平，更有用的结果相比，单独使用SET。
Abstract Scholarly debate about student evaluations of teaching (SETs) often focuses on whether SETs are valid, reliable and unbiased. In this article, we assume the most optimistic conditions for SETs that are supported by the empirical literature. Specifically, we assume that SETs are moderately correlated with teaching quality (student learning and instructional best practices), highly reliable, and do not systematically discriminate on any instructionally irrelevant basis. We use computational simulation to show that, under ideal circumstances, even careful and judicious use of SETs to assess faculty can produce an unacceptably high error rate: (a) a large difference in SET scores fails to reliably identify the best teacher in a pairwise comparison, and (b) more than a quarter of faculty with evaluations at or below the 20th percentile are above the median in instructional quality. These problems are attributable to imprecision in the relationship between SETs and instructor quality that exists even when they are moderately correlated. Our simulation indicates that evaluating instruction using multiple imperfect measures, including but not limited to SETs, can produce a fairer and more useful result compared to using SETs alone.