Collaborative Research: Exploring Algorithmic Fairness and Potential Bias in K-12 Mathematics Adaptive Learning

协作研究：探索 K-12 数学自适应学习中的算法公平性和潜在偏差

• 批准号: 2000638
• 负责人: Nigel Bosch
• 金额: $ 98.7万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2000638&HistoricalAwards=false
• 关键词: Collaborative; Research; Exploring; Algorithmic; Fairness

Students in middle school and high school often use adaptive learning software as part of their math education experience. Adaptive learning software works by automatically measuring how much students have learned about the topic, as well as their learning process and experiences, and then adjusting the instruction accordingly. This project will investigate potential ways in which adaptive learning software might be biased against students from certain groups, and how such biases can be reduced. Adaptive learning offers an opportunity to provide high quality instruction that is personalized to the needs of individual learners, but little is known about who benefits most from adaptive learning technologies. This project will address this issue by observing and interviewing students who use adaptive math learning software to discover what aspects of their identity are most salient in the adaptive learning context. This project will then investigate possible algorithmic biases related to the identities that students express. Findings from the project will contribute to understanding of the most relevant aspects of student identity in adaptive learning contexts, and how those identities affect their learning experience. Finally, this project will address the biases that are identified, thereby providing a more equitable mathematics education experience for students. Modern adaptive learning platforms individualize learning support and improve learner outcomes by using algorithms that are typically derived through machine learning. Previous work has studied biases in educational model accuracy for large groups (e.g., ethnic and gendered categories, urban vs. rural, etc.); however, large groups may have a great deal of heterogeneity, and little is known about which specific groups of students suffer from biases in model accuracy and why. This project will approach the problem of potential bias in three steps. First, the project will begin by collecting data on educational software usage patterns (i.e., logs of actions and classroom observations of student experiences) for students using MATHia, a math education platform used by over half a million students across the United States. As part of this data collection, students will describe their identity in open-ended survey responses and interviews, which will be analyzed to discover identity characteristics that shape their learning experiences. Second, existing machine learning models will be applied to these data to predict knowledge, engagement, and self-regulated learning behaviors, and the predictions will be analyzed to reveal cases where models are systematically biased. Third, the project will compare various pre-processing, in-processing, and post- processing methods for bias reduction, and study the effects of the improved algorithms when applied in MATHia. Results from this project will contribute to scientific understanding of the role of student identity in adaptive learning software, biases in machine learning for educational software, and the effects of applying machine learning methods for bias reduction. This project is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development, with co-funding by the Discovery Research PreK-12 (DRK-12) program.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

初中生和高中生经常使用自适应学习软件作为他们数学教育体验的一部分。适应性学习软件的工作原理是自动测量学生对该主题的了解程度，以及他们的学习过程和经验，然后相应地调整教学。这个项目将调查自适应学习软件可能对某些群体的学生产生偏见的潜在方式，以及如何减少这种偏见。适应性学习提供了提供针对个别学习者需求的高质量教学的机会，但对于谁从适应性学习技术中受益最大却知之甚少。这个项目将通过观察和采访使用适应性数学学习软件的学生来解决这个问题，以发现他们身份的哪些方面在适应性学习环境中最突出。这个项目随后将调查与学生表达的身份相关的可能的算法偏差。该项目的发现将有助于理解适应性学习环境中学生身份最相关的方面，以及这些身份如何影响他们的学习体验。最后，这个项目将解决已确定的偏见，从而为学生提供更公平的数学教育体验。现代自适应学习平台通过使用通常通过机器学习派生的算法，使学习支持个性化，并改善学习者的结果。以前的工作已经研究了大群体(例如，种族和性别类别、城市和农村等)在教育模型准确性方面的偏差；然而，大群体可能具有很大的异质性，对于哪些特定的学生群体在模型准确性方面存在偏差及其原因知之甚少。这个项目将分三个步骤解决潜在的偏见问题。首先，该项目将从收集学生使用Mathia的教育软件使用模式(即行动日志和学生体验的课堂观察)的数据开始。Mathia是一个数学教育平台，全美有50多万学生使用。作为数据收集的一部分，学生将在开放式调查回复和访谈中描述他们的身份，这些调查将被分析，以发现塑造他们学习体验的身份特征。其次，现有的机器学习模型将应用于这些数据，以预测知识、参与度和自我调节学习行为，并将对预测进行分析，以揭示模型存在系统性偏差的情况。第三，该项目将比较各种前处理、中处理和后处理方法来减少偏差，并研究改进算法在Mathia中的应用效果。这个项目的结果将有助于科学地理解学生身份在自适应学习软件中的作用，在教育软件的机器学习中的偏见，以及应用机器学习方法减少偏见的效果。该项目由EHR核心研究(ECR)计划支持，该计划支持推进STEM学习和学习环境的基础研究、扩大对STEM的参与以及STEM劳动力发展的工作，并由发现研究PreK-12(DRK-12)计划共同资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Generalizing Predictive Models of Reading Ability in Adaptive Mathematics Software

自适应数学软件中阅读能力预测模型的推广

• 发表时间: 2023
• 期刊: Proceedings of the 16th International Conference on Educational Data Mining
• 作者: Almoubayyed, Husni;Fancsali, Stephen;Ritter, Steve
• 通讯作者: Ritter, Steve

Constructing categories: Moving beyond protected classes in algorithmic fairness

构建类别：在算法公平性方面超越受保护类别

• DOI: 10.1002/asi.24643
• 发表时间: 2022
• 期刊: Journal of the Association for Information Science and Technology
• 影响因子: 3.5
• 作者: Belitz, Clara;Ocumpaugh, Jaclyn;Ritter, Steven;Baker, Ryan S.;Fancsali, Stephen E.;Bosch, Nigel
• 通讯作者: Bosch, Nigel

Evaluating Gaming Detector Model Robustness Over Time

评估游戏探测器模型随时间的稳健性

• DOI: 10.5281/zenodo.6852961
• 发表时间: 2022
• 期刊: International Educational Data Mining Society
• 作者: Levin, Nathan;Baker, Ryan;Nasiar, Nidhi;Fancsali Stephen;Hutt, Stephen
• 通讯作者: Hutt, Stephen