Examining the Potential and Limits of Rubrics as Tools for Racial Equity in Graduate Admissions: A Sequential Mixed-Methods Inquiry

大学如何做出影响高等教育机会的决定是科学、技术、工程和数学（STEM）领域种族、性别和社会经济不平等的根本原因。特别是考虑到在录取中考虑种族的法律环境发生了变化，迫切需要开发工具来减少录取决策中的种族偏见。越来越多的高等教育机构在招生中采用标准，尤其是在可选考试和免考试成为新常态的情况下。研究表明，标准可以减少K-12教师的种族偏见，提高教师招聘的多样性。然而，几乎没有证据表明在研究生招生中应该如何设计或实施标准。特别需要两种类型的证据：1)关于各种学科和机构类型中规则有用性的概括数据；2)关于规则可行性的生活经验以及在评估申请人时它们所提示的潜在假设。因此，该项目的目标是扩大关于研究生招生规则的证据基础，包括多种数据类型和多层次的分析。它有一个连续的混合方法研究设计，涉及大规模实验和与已建立的国家研究实践伙伴关系成员进行定性研究。具体来说，它旨在从三个层面推进对STEM博士录取中种族平等评估标准的潜力和局限性的认识：1)标准是否以及如何影响个人层面的种族偏见；2)组织使用什么标准实践进行选择，以及他们如何克服使用标准的障碍；3)关于卓越的文化价值观常常驱动个人判断和组织实践。该项目意义重大，因为它是第一个关于如何在STEM博士课程中设计和使用规则的实验研究。它旨在为相关评估环境的讨论和实践提供信息，包括奖学金/奖学金、奖励、招聘和终身教职/晋升。本项目由美国国家科学基金会EDU核心研究（ECR）项目支持。ECR项目强调在该领域产生基础知识的基础STEM教育研究。投资在至关重要、广泛和持久的关键领域：STEM学习和STEM学习环境，扩大STEM参与，以及STEM劳动力发展。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

How universities make the decisions that affect access to higher education are a root cause of racial, gender, and socioeconomic inequalities in science, technology, engineering, and mathematics (STEM). Particularly given changes to the legal landscape for considering race in admissions, it is urgent to develop tools that reduce racial bias in admissions decisions. More higher education institutions are adopting rubrics in admissions, particularly as test-optional and test-free become the new normal, and research suggests that rubrics reduce K-12 teacher racial bias and improve diversity in faculty hiring. However, there is little evidence about how rubrics should be designed or implemented in graduate admissions. Two types of evidence are needed in particular: 1) Generalizable data about the usefulness of rubrics in a variety of disciplinary and institutional types and 2) Lived experiences about the feasibility of rubrics and the underlying assumptions they prompt when evaluating applicants.The goal of this project is therefore to expand the evidence base about rubrics in graduate admissions to include multiple data types and multiple levels of analysis. It has a sequential mixed methods research design involving a large-scale experiment and qualitative research with members of established, national research-practice partnerships. Specifically, it aims to advance knowledge about the potential and limits of evaluation rubrics for racial equity in STEM PhD admissions at three levels: 1) whether and how rubrics affect racial biases at the individual level; 2) what standard practices organizations use for selection and how they navigate barriers to using rubrics; and 3) the cultural values about excellence that often drive individual judgment and organizational practices. The project is significant as the first experimental study of how rubrics can be designed and used in STEM PhD programs. It aims to inform discussion and practice in adjacent evaluation contexts, including fellowships/ scholarships, awards, hiring, and tenure/ promotion.This project is supported by NSF's EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.