Examining the Potential and Limits of Rubrics as Tools for Racial Equity in Graduate Admissions: A Sequential Mixed-Methods Inquiry

大学如何做出影响接受高等教育的决定是科学，技术，工程和数学（STEM）中种族，性别和社会经济不平等的根本原因。特别是考虑到在招生中考虑种族的法律的环境发生了变化，迫切需要开发减少招生决定中种族偏见的工具。越来越多的高等教育机构在招生中采用分数线，特别是随着考试可选和免试成为新常态，研究表明分数线减少了K-12教师的种族偏见，提高了教师招聘的多样性。然而，很少有证据表明，在研究生招生中应该如何设计或实施评分标准。特别需要两种类型的证据：1）在各种学科和机构类型中关于评分标准有用性的可推广数据; 2）关于评分标准可行性的生活经验以及在评估申请人时它们所提示的潜在假设。因此，本项目的目标是扩展研究生招生中关于评分标准的证据基础，以包括多种数据类型和多层次的分析。它有一个连续的混合方法研究设计，涉及大规模的实验和定性研究与既定的国家研究实践伙伴关系的成员。具体而言，它旨在在三个层面上促进对STEM博士招生中种族平等评估规则的潜力和限制的了解：1）规则是否以及如何影响个人层面的种族偏见; 2）组织用于选择的标准实践以及他们如何克服使用规则的障碍;以及3）关于卓越的文化价值观，这些价值观通常会推动个人判断和组织实践。该项目是重要的，因为它是第一个关于如何在STEM博士课程中设计和使用标题的实验研究。它的目的是通知讨论和实践在相邻的评价contextuals. This项目的支持下，包括奖学金/奖学金，奖励，雇用，和终身/ promotion.This项目的核心研究（ECR）计划。ECR计划强调基础STEM教育研究，产生该领域的基础知识。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响力审查标准进行评估，被认为值得支持。

How universities make the decisions that affect access to higher education are a root cause of racial, gender, and socioeconomic inequalities in science, technology, engineering, and mathematics (STEM). Particularly given changes to the legal landscape for considering race in admissions, it is urgent to develop tools that reduce racial bias in admissions decisions. More higher education institutions are adopting rubrics in admissions, particularly as test-optional and test-free become the new normal, and research suggests that rubrics reduce K-12 teacher racial bias and improve diversity in faculty hiring. However, there is little evidence about how rubrics should be designed or implemented in graduate admissions. Two types of evidence are needed in particular: 1) Generalizable data about the usefulness of rubrics in a variety of disciplinary and institutional types and 2) Lived experiences about the feasibility of rubrics and the underlying assumptions they prompt when evaluating applicants.The goal of this project is therefore to expand the evidence base about rubrics in graduate admissions to include multiple data types and multiple levels of analysis. It has a sequential mixed methods research design involving a large-scale experiment and qualitative research with members of established, national research-practice partnerships. Specifically, it aims to advance knowledge about the potential and limits of evaluation rubrics for racial equity in STEM PhD admissions at three levels: 1) whether and how rubrics affect racial biases at the individual level; 2) what standard practices organizations use for selection and how they navigate barriers to using rubrics; and 3) the cultural values about excellence that often drive individual judgment and organizational practices. The project is significant as the first experimental study of how rubrics can be designed and used in STEM PhD programs. It aims to inform discussion and practice in adjacent evaluation contexts, including fellowships/ scholarships, awards, hiring, and tenure/ promotion.This project is supported by NSF's EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.