Collaborative Research: Exploring Algorithmic Fairness and Potential Bias in K-12 Mathematics Adaptive Learning

初中和高中的学生经常使用自适应学习软件作为他们数学教育经验的一部分。自适应学习软件的工作原理是自动测量学生对该主题的了解程度，以及他们的学习过程和经验，然后相应地调整教学。该项目将调查自适应学习软件可能对某些群体的学生有偏见的潜在方式，以及如何减少这种偏见。自适应学习提供了一个机会，提供高质量的教学，个性化的个人学习者的需求，但很少有人知道谁受益最多的自适应学习技术。这个项目将通过观察和采访使用自适应数学学习软件的学生来解决这个问题，以发现他们的身份的哪些方面在自适应学习环境中最突出。然后，该项目将调查与学生表达的身份有关的可能的算法偏见。该项目的研究结果将有助于了解学生身份在适应性学习环境中最相关的方面，以及这些身份如何影响他们的学习体验。最后，该项目将解决已确定的偏见，从而为学生提供更公平的数学教育体验。 现代自适应学习平台通过使用通常通过机器学习获得的算法来个性化学习支持并提高学习者的成果。以前的工作已经研究了大群体教育模型准确性的偏差（例如，种族和性别类别、城市与农村等）;然而，大的群体可能有很大的异质性，而且很少有人知道哪些特定的学生群体在模型准确性方面存在偏差，以及为什么。本项目将分三步解决潜在偏见问题。首先，该项目将开始收集教育软件使用模式的数据（即，行动日志和学生体验的课堂观察）为使用MATHia的学生提供服务，MATHia是一个数学教育平台，在美国有超过50万学生使用。作为数据收集的一部分，学生将在开放式调查问卷和访谈中描述他们的身份，并对其进行分析，以发现塑造他们学习经历的身份特征。其次，现有的机器学习模型将应用于这些数据，以预测知识，参与和自我调节的学习行为，并将对预测进行分析，以揭示模型存在系统性偏差的情况。第三，本项目将比较各种预处理、处理中和后处理方法以减少偏差，并研究改进算法在MATHia中应用时的效果。该项目的结果将有助于科学地理解学生身份在自适应学习软件中的作用，教育软件机器学习中的偏见，以及应用机器学习方法减少偏见的效果。 该项目得到了EHR核心研究（ECR）计划的支持，该计划支持推进STEM学习和学习环境的基础研究，扩大STEM参与和STEM劳动力发展的工作。由Discovery Research PreK-12（DRK-12）共同资助该奖项反映了NSF的法定使命，并通过使用基金会的智力价值进行评估，被认为值得支持和更广泛的影响审查标准。

Students in middle school and high school often use adaptive learning software as part of their math education experience. Adaptive learning software works by automatically measuring how much students have learned about the topic, as well as their learning process and experiences, and then adjusting the instruction accordingly. This project will investigate potential ways in which adaptive learning software might be biased against students from certain groups, and how such biases can be reduced. Adaptive learning offers an opportunity to provide high quality instruction that is personalized to the needs of individual learners, but little is known about who benefits most from adaptive learning technologies. This project will address this issue by observing and interviewing students who use adaptive math learning software to discover what aspects of their identity are most salient in the adaptive learning context. This project will then investigate possible algorithmic biases related to the identities that students express. Findings from the project will contribute to understanding of the most relevant aspects of student identity in adaptive learning contexts, and how those identities affect their learning experience. Finally, this project will address the biases that are identified, thereby providing a more equitable mathematics education experience for students. Modern adaptive learning platforms individualize learning support and improve learner outcomes by using algorithms that are typically derived through machine learning. Previous work has studied biases in educational model accuracy for large groups (e.g., ethnic and gendered categories, urban vs. rural, etc.); however, large groups may have a great deal of heterogeneity, and little is known about which specific groups of students suffer from biases in model accuracy and why. This project will approach the problem of potential bias in three steps. First, the project will begin by collecting data on educational software usage patterns (i.e., logs of actions and classroom observations of student experiences) for students using MATHia, a math education platform used by over half a million students across the United States. As part of this data collection, students will describe their identity in open-ended survey responses and interviews, which will be analyzed to discover identity characteristics that shape their learning experiences. Second, existing machine learning models will be applied to these data to predict knowledge, engagement, and self-regulated learning behaviors, and the predictions will be analyzed to reveal cases where models are systematically biased. Third, the project will compare various pre-processing, in-processing, and post- processing methods for bias reduction, and study the effects of the improved algorithms when applied in MATHia. Results from this project will contribute to scientific understanding of the role of student identity in adaptive learning software, biases in machine learning for educational software, and the effects of applying machine learning methods for bias reduction. This project is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development, with co-funding by the Discovery Research PreK-12 (DRK-12) program.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.