Examining the Potential and Limits of Rubrics as Tools for Racial Equity in Graduate Admissions: A Sequential Mixed-Methods Inquiry

大学如何做出影响接受高等教育机会的决定，是科学、技术、工程和数学(STEM)领域种族、性别和社会经济不平等的根本原因。特别是考虑到在招生中考虑种族的法律环境发生了变化，迫切需要开发工具来减少招生决定中的种族偏见。越来越多的高等教育机构在招生中采用校规，特别是在可选考试和免试成为新常态的情况下，研究表明，校规减少了K-12教师的种族偏见，并提高了教师招聘的多样性。然而，几乎没有证据表明，在研究生招生中应该如何设计或实施卢布规则。特别需要两种类型的证据：1)关于在各种学科和机构类型中使用Rubrics的概括性数据，以及2)关于Rubrics的可行性及其在评估申请者时提出的潜在假设的生活经验。因此，本项目的目标是扩大研究生招生中Rubrics的证据基础，使其包括多种数据类型和多层次的分析。它采用顺序混合方法的研究设计，包括与已建立的国家研究-实践伙伴关系成员进行大规模实验和定性研究。具体地说，它的目的是在三个层面上促进人们对STEM博士招生中种族公平评估标准的潜在和限制的了解：1)标准标准是否以及如何在个人层面上影响种族偏见；2)组织在选择时使用什么标准做法，以及他们如何克服使用标准标准的障碍；以及3)关于优秀的文化价值观，通常会推动个人判断和组织实践。该项目是关于如何在STEM博士项目中设计和使用Rubrics的第一个实验研究，具有重要意义。它旨在为相邻评估环境中的讨论和实践提供信息，包括奖学金/奖学金、奖项、招聘和终身教职/晋升。该项目由NSF的EDU核心研究(ECR)计划支持。ECR计划强调基础STEM教育研究，以产生该领域的基础知识。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，认为值得支持。

How universities make the decisions that affect access to higher education are a root cause of racial, gender, and socioeconomic inequalities in science, technology, engineering, and mathematics (STEM). Particularly given changes to the legal landscape for considering race in admissions, it is urgent to develop tools that reduce racial bias in admissions decisions. More higher education institutions are adopting rubrics in admissions, particularly as test-optional and test-free become the new normal, and research suggests that rubrics reduce K-12 teacher racial bias and improve diversity in faculty hiring. However, there is little evidence about how rubrics should be designed or implemented in graduate admissions. Two types of evidence are needed in particular: 1) Generalizable data about the usefulness of rubrics in a variety of disciplinary and institutional types and 2) Lived experiences about the feasibility of rubrics and the underlying assumptions they prompt when evaluating applicants.The goal of this project is therefore to expand the evidence base about rubrics in graduate admissions to include multiple data types and multiple levels of analysis. It has a sequential mixed methods research design involving a large-scale experiment and qualitative research with members of established, national research-practice partnerships. Specifically, it aims to advance knowledge about the potential and limits of evaluation rubrics for racial equity in STEM PhD admissions at three levels: 1) whether and how rubrics affect racial biases at the individual level; 2) what standard practices organizations use for selection and how they navigate barriers to using rubrics; and 3) the cultural values about excellence that often drive individual judgment and organizational practices. The project is significant as the first experimental study of how rubrics can be designed and used in STEM PhD programs. It aims to inform discussion and practice in adjacent evaluation contexts, including fellowships/ scholarships, awards, hiring, and tenure/ promotion.This project is supported by NSF's EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.