CAREER: Fair assessments: Examining Cultural Familiarity to Decrease Bias in Engineering Classroom Assessments

职业：公平评估：检查文化熟悉程度以减少工程课堂评估中的偏见

• 批准号: 2047420
• 负责人: Kerrie Douglas
• 金额: $ 51.35万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-03-01 至 2026-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2047420&HistoricalAwards=false
• 关键词: CAREER; Fair; assessments; Examining; Cultural

Test scores are used to inform who gets to pursue and stay in engineering education at every juncture, pre-college through graduate school. The persistent achievement discrepancies between students of Color and White undergraduate engineering students are commonly blamed on students or their previous education rather than acknowledging that the tests themselves or testing process could be biased against test takers. Factors such as differing cultures, educational access, and test taking strategies all contribute to the test taking experience and can result in tests that are inherently easier or harder for different groups of students. Very little is known about how biases contribute to the problem of discrepancies in achievement between students of Color and their White peers in engineering. The research on test fairness in engineering education has been limited by the low representation of students of Color in engineering. This CAREER project will apply an innovative research approach for small sample sizes that will advance understanding about the role of cultural familiarity within the context of concept inventories. By answering the research questions, this CAREER project will build a firm foundation for a life-long leadership position in research on fair assessment practices and educational activities to train instructors in increasing the fairness of their classroom tests. This CAREER project will inform the field of engineering education by identifying how engineering education instructors and researchers can increase the fairness of assessments. Through its integrated research and education plans, the work will improve decision-making in engineering education and achievement for students of Color. Because there are so few students of Color in engineering, traditional approaches to studying assessment bias cannot be used. This project will use an explanatory sequential mixed-methods approach which integrates studies of item difficulty and discrimination for each group of students separately and then will conduct interviews to explain the quantitative findings and build theory. The new approaches and methods developed will lead to significant strides in solving the problem of unfair assessment in engineering education. The research questions will investigate: To what extent do items in these three assessments demonstrate adequate functioning for students of Color? To what extent do score discrepancies remain once inadequate items are removed? What are patterns of cultural familiarity and content of problematic items and items that show acceptable functioning? How do students of Color experience testing in concept inventories? In the education plan, findings will be applied to develop and disseminate educational materials for three primary groups of learners: (1) future engineering faculty, (2) current engineering faculty, and (3) minority- and women-in-engineering program staff. The findings will benefit society directly from the research outcomes, activities related to conducting the research in this project, and educational activities. The first research question provides important information concerning the extent to which unfairness in assessments contributes to disparities in achievement. The second and third research questions lead to principles for fair assessments that will be widely shared with engineering instructors and future faculty (via a graduate course and partnerships with national engineering education organizations); these will improve fairness for thousands of students across the country.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

考试成绩被用来通知谁可以在每一个关键时刻继续接受工程教育，从大学预科到研究生院。有色人种学生和白色本科工程专业学生之间持续的成绩差异通常归咎于学生或他们以前的教育，而不是承认考试本身或考试过程可能对考生有偏见。不同的文化，教育机会和考试策略等因素都有助于考试体验，并可能导致考试对不同学生群体来说本质上更容易或更难。很少有人知道如何偏见有助于之间的差异问题的学生的颜色和他们的白色同龄人在工程。有色人种学生在工科专业中的代表性较低，限制了工科教育考试公平性的研究。这个CAREER项目将采用一种创新的小样本研究方法，这将促进对概念库存背景下文化熟悉度的作用的理解。通过回答研究问题，这个职业生涯项目将建立一个坚实的基础，在公平评估实践和教育活动的研究终身领导地位，以培训教师在增加他们的课堂测试的公平性。这个CAREER项目将通过确定工程教育教师和研究人员如何提高评估的公平性来告知工程教育领域。通过其综合研究和教育计划，这项工作将改善工程教育的决策和学生的颜色的成就。由于工程专业的色彩专业学生很少，因此无法使用传统的方法来研究评估偏差。本研究将采用解释性序贯混合方法，将项目难度和区分度的研究分别整合到每组学生中，然后进行访谈，解释定量研究结果并建立理论。新的方法和途径的开发将导致在解决工程教育中的不公平评估的问题的重大进展。研究问题将调查：在多大程度上做这三个评估项目证明足够的功能，为学生的颜色？在何种程度上，分数差异仍然存在，一旦不充分的项目被删除？问题项目和表现出可接受功能的项目的文化熟悉模式和内容是什么？色彩专业的学生如何体验概念量表的测试？在教育计划中，调查结果将用于为三个主要学习者群体开发和传播教育材料：（1）未来的工程学院，（2）当前的工程学院，（3）少数民族和女性工程项目工作人员。研究结果将使社会直接受益于研究成果、与开展本项目研究有关的活动以及教育活动。第一个研究问题提供了重要的信息，说明评估中的不公平在多大程度上导致了成绩的差异。第二个和第三个研究问题导致公平评估的原则，将与工程教师和未来的教师广泛分享（通过研究生课程和与国家工程教育组织的伙伴关系）;这一奖项反映了NSF的法定使命，并被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准。

项目成果

The Assessment Dilemma: Examining our practices through the lens of equity and fairness

评估困境：从公平和公正的角度审视我们的做法

• DOI: 10.1109/fie56618.2022.9962378
• 发表时间: 2022
• 期刊: IEEE
• 作者: Douglas, Kerrie A.;Cross, Kelly J.;Purzer, Senay
• 通讯作者: Purzer, Senay

Work In Progress: Evaluating the Cultural Context of Engineering and Engineering-Related Concept Inventory Assessment Items

正在进行的工作：评估工程和工程相关概念清单评估项目的文化背景

• DOI: 10.18260/1-2--44246
• 发表时间: 2023
• 期刊: ASEE Conferences
• 作者: Adams, Shauna;Douglas, Kerrie;Goldberg, Philip
• 通讯作者: Goldberg, Philip