Collaborative Research: Exploring Algorithmic Fairness and Potential Bias in K-12 Mathematics Adaptive Learning

初高中的学生经常使用适应性学习软件作为他们数学教育经验的一部分。自适应学习软件的工作原理是自动测量学生对主题的了解程度，以及他们的学习过程和经验，然后相应地调整教学。这个项目将调查适应性学习软件可能对某些群体的学生产生偏见的潜在方式，以及如何减少这种偏见。适应性学习为提供高质量的教学提供了一个机会，这种教学是针对个体学习者的需求进行个性化的，但是对于谁从适应性学习技术中受益最大却知之甚少。本项目将通过观察和采访使用适应性数学学习软件的学生来解决这个问题，以发现他们身份的哪些方面在适应性学习环境中最突出。然后，该项目将调查与学生表达的身份相关的可能的算法偏差。该项目的研究结果将有助于理解适应性学习环境中学生身份的最相关方面，以及这些身份如何影响他们的学习体验。最后，本项目将解决已确定的偏见，从而为学生提供更公平的数学教育体验。现代自适应学习平台通过使用通常由机器学习衍生的算法来个性化学习支持并提高学习者的学习成果。以前的工作研究了大群体（例如，种族和性别类别，城市与农村等）教育模型准确性的偏差；然而，大的群体可能有很大的异质性，并且很少知道哪些特定的学生群体在模型准确性方面存在偏差以及原因。这个项目将分三步来解决潜在偏见的问题。首先，该项目将首先为使用MATHia的学生收集有关教育软件使用模式的数据（即，行为日志和学生体验的课堂观察），MATHia是美国50多万学生使用的数学教育平台。作为数据收集的一部分，学生将在开放式调查和访谈中描述他们的身份，这些调查和访谈将被分析以发现塑造他们学习经历的身份特征。其次，将现有的机器学习模型应用于这些数据，以预测知识、参与度和自我调节的学习行为，并对预测进行分析，以揭示模型系统性偏差的情况。第三，本项目将比较各种减少偏差的预处理、处理中和后处理方法，并研究改进算法在MATHia中的应用效果。这个项目的结果将有助于科学地理解学生身份在自适应学习软件中的作用，教育软件中机器学习的偏见，以及应用机器学习方法减少偏见的影响。该项目由EHR核心研究（ECR）计划支持，该计划支持推进STEM学习和学习环境的基础研究，扩大STEM参与和STEM劳动力发展的工作，并由发现研究PreK-12 （DRK-12）计划共同资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Students in middle school and high school often use adaptive learning software as part of their math education experience. Adaptive learning software works by automatically measuring how much students have learned about the topic, as well as their learning process and experiences, and then adjusting the instruction accordingly. This project will investigate potential ways in which adaptive learning software might be biased against students from certain groups, and how such biases can be reduced. Adaptive learning offers an opportunity to provide high quality instruction that is personalized to the needs of individual learners, but little is known about who benefits most from adaptive learning technologies. This project will address this issue by observing and interviewing students who use adaptive math learning software to discover what aspects of their identity are most salient in the adaptive learning context. This project will then investigate possible algorithmic biases related to the identities that students express. Findings from the project will contribute to understanding of the most relevant aspects of student identity in adaptive learning contexts, and how those identities affect their learning experience. Finally, this project will address the biases that are identified, thereby providing a more equitable mathematics education experience for students. Modern adaptive learning platforms individualize learning support and improve learner outcomes by using algorithms that are typically derived through machine learning. Previous work has studied biases in educational model accuracy for large groups (e.g., ethnic and gendered categories, urban vs. rural, etc.); however, large groups may have a great deal of heterogeneity, and little is known about which specific groups of students suffer from biases in model accuracy and why. This project will approach the problem of potential bias in three steps. First, the project will begin by collecting data on educational software usage patterns (i.e., logs of actions and classroom observations of student experiences) for students using MATHia, a math education platform used by over half a million students across the United States. As part of this data collection, students will describe their identity in open-ended survey responses and interviews, which will be analyzed to discover identity characteristics that shape their learning experiences. Second, existing machine learning models will be applied to these data to predict knowledge, engagement, and self-regulated learning behaviors, and the predictions will be analyzed to reveal cases where models are systematically biased. Third, the project will compare various pre-processing, in-processing, and post- processing methods for bias reduction, and study the effects of the improved algorithms when applied in MATHia. Results from this project will contribute to scientific understanding of the role of student identity in adaptive learning software, biases in machine learning for educational software, and the effects of applying machine learning methods for bias reduction. This project is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development, with co-funding by the Discovery Research PreK-12 (DRK-12) program.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.