CAREER: Fair assessments: Examining Cultural Familiarity to Decrease Bias in Engineering Classroom Assessments

考试成绩被用来通知谁可以在大学前到研究生院的每一个关头继续和留在工程教育中。有色人种学生和白人工科本科生之间持续的成绩差异通常被归咎于学生或他们之前的教育，而不是承认考试本身或测试过程可能对考生有偏见。不同的文化、教育机会和考试策略等因素都有助于考试体验，并可能导致考试对不同的学生群体来说天生更容易或更难。关于偏见如何导致有色人种学生和他们的白人同龄人在工程学上的成绩差异的问题，我们知之甚少。由于工科有色人种学生代表性较低，限制了工科教育考试公平性的研究。这一职业项目将对小样本采用一种创新的研究方法，这将促进对文化熟悉在概念清单中的作用的理解。通过回答研究问题，这个职业项目将为在公平评估实践和教育活动方面的终身领导地位奠定坚实的基础，以培训教师提高课堂测试的公平性。这个职业项目将通过确定工程教育教师和研究人员如何提高评估的公正性来向工程教育领域提供信息。通过其综合研究和教育计划，这项工作将改善工程教育的决策和颜色学生的成绩。由于工科色彩专业的学生太少，传统的方法不能用来研究评估偏差。本项目将使用解释性顺序混合方法，分别对每一组学生进行项目难度和辨别力的研究，然后进行访谈，解释量化结果并建立理论。这些新的途径和方法将在解决工科教育中的不公平评价问题上取得重大进展。研究问题将调查：这三个评估中的项目在多大程度上证明对有色人种学生来说是足够的功能？一旦剔除不充分的项目，分数差异会保持到什么程度？有问题的物品和表现出可接受功能的物品的文化熟悉模式和内容是什么？色彩专业的学生如何体验概念清单中的测试？在教育计划中，研究结果将被用于为三个主要的学习者群体开发和传播教育材料：(1)未来的工程教师，(2)现有的工程教师，(3)少数族裔和女性工程项目工作人员。研究结果将直接从研究成果、与开展本项目研究相关的活动以及教育活动中受益。第一个研究问题提供了关于评估中的不公平在多大程度上造成成绩差距的重要信息。第二个和第三个研究问题导致了公平评估的原则，这些原则将被广泛分享给工程教师和未来的教师(通过研究生课程和与国家工程教育组织的合作)；这些将改善全国数千名学生的公平。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Test scores are used to inform who gets to pursue and stay in engineering education at every juncture, pre-college through graduate school. The persistent achievement discrepancies between students of Color and White undergraduate engineering students are commonly blamed on students or their previous education rather than acknowledging that the tests themselves or testing process could be biased against test takers. Factors such as differing cultures, educational access, and test taking strategies all contribute to the test taking experience and can result in tests that are inherently easier or harder for different groups of students. Very little is known about how biases contribute to the problem of discrepancies in achievement between students of Color and their White peers in engineering. The research on test fairness in engineering education has been limited by the low representation of students of Color in engineering. This CAREER project will apply an innovative research approach for small sample sizes that will advance understanding about the role of cultural familiarity within the context of concept inventories. By answering the research questions, this CAREER project will build a firm foundation for a life-long leadership position in research on fair assessment practices and educational activities to train instructors in increasing the fairness of their classroom tests. This CAREER project will inform the field of engineering education by identifying how engineering education instructors and researchers can increase the fairness of assessments. Through its integrated research and education plans, the work will improve decision-making in engineering education and achievement for students of Color. Because there are so few students of Color in engineering, traditional approaches to studying assessment bias cannot be used. This project will use an explanatory sequential mixed-methods approach which integrates studies of item difficulty and discrimination for each group of students separately and then will conduct interviews to explain the quantitative findings and build theory. The new approaches and methods developed will lead to significant strides in solving the problem of unfair assessment in engineering education. The research questions will investigate: To what extent do items in these three assessments demonstrate adequate functioning for students of Color? To what extent do score discrepancies remain once inadequate items are removed? What are patterns of cultural familiarity and content of problematic items and items that show acceptable functioning? How do students of Color experience testing in concept inventories? In the education plan, findings will be applied to develop and disseminate educational materials for three primary groups of learners: (1) future engineering faculty, (2) current engineering faculty, and (3) minority- and women-in-engineering program staff. The findings will benefit society directly from the research outcomes, activities related to conducting the research in this project, and educational activities. The first research question provides important information concerning the extent to which unfairness in assessments contributes to disparities in achievement. The second and third research questions lead to principles for fair assessments that will be widely shared with engineering instructors and future faculty (via a graduate course and partnerships with national engineering education organizations); these will improve fairness for thousands of students across the country.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.