FairFL-MC: A Metacognitive Calibration Intervention Powered by Fair and Private Machine Learning

学生往往很难估计自己的知识水平。该项目的目标是研究如何提高学生估计自己知识的能力，使用一个由短期训练练习组成的学生支持系统，该训练练习将使用人工智能(AI)方法进行个性化。虽然在教育背景下对人工智能方法进行了大量研究，但此类项目很少考虑广泛采用个性化教育软件所需的一些关键社会和人类因素，如隐私和公平。该项目通过一个专门针对教育背景的新型去中心化人工智能框架来解决这些问题。该项目框架将使研究人员能够创建人工智能系统，作为培训练习的一部分向学生提供反馈，所有这些都不需要直接访问他们的数据，同时还可以训练人工智能系统减少与学生身份的关键方面相关的偏见，如他们的人口统计数据。培训练习将包括教育活动，学生评估他们的考试成绩，从人工智能系统接收反馈，并反思他们的知识。该项目框架的隐私和公平能力将改变中学后在线学习，后者有望受益于新兴的人工智能驱动的学习技术，但尚未完全实现这些好处。该项目将直接使参与研究的学生受益，因为他们将提高知识估计技能，更有效地为课堂测试做准备，并了解潜在的侵犯隐私和人工智能偏见。鉴于该项目的公平重点，研究团队将特别关注STEM(科学、技术、工程和数学)中传统上代表性较低的群体的学生的利益，确保人工智能支持的框架对他们同样有帮助，并确保他们对隐私和公平的观点受到特别关注。该项目将通过纳入对学生数据的严格隐私保障和跨多个学生人口统计群体的公平考虑来推动人工智能研究。此外，它还将通过确定先发制人反馈对提高知识估计技能的有效性来推动教育研究，并将研究先发制人改善知识估计影响学术成果的机制。特别是，该项目将通过在学习科学和技术方面的跨学科创新实现四个研究目标。首先，该团队将确定通过人工智能支持的先发制人反馈可以在多大程度上改善学生的元认知校准，学生可能会感觉到与事后反馈不同。第二，该项目将通过研究项目中干预的工作机制来扩大对元认知校准和校准干预的理论理解。第三，该团队将通过创新的联合学习模型解决人工智能模型的公平性和准确性之间的根本权衡。第四，该团队将在现实世界的教育数据集上评估人工智能框架，并将其性能与最先进的基线进行比较，以保护隐私和减轻偏见。项目团队将通过研讨会、出版物和互动活动来传播项目成果，并将在整个项目中培训来自不同背景的本科生和研究生。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Students often have difficulty estimating their own level of knowledge. The goal of this project is to research ways to improve students' ability to estimate their knowledge, using a student support system consisting of short training exercises that will be personalized with artificial intelligence (AI) methods. While there is abundant research on AI methods in educational contexts, such projects rarely consider some of the key social and human factors, such as privacy and fairness, that are needed for widespread adoption of personalized educational software. This project addresses these issues with a novel decentralized AI framework that is specifically for education contexts. The project framework will enable researchers to create AI systems that provide feedback to students as part of their training exercises, all without directly accessing their data and while also training the AI system to reduce biases related to key aspects of students' identity, such as their demographics. The training exercises will include educational activities where students estimate their test scores, receive feedback from the AI system, and reflect on their knowledge. The privacy and fairness capabilities of the project framework will transform postsecondary online learning, which is poised to benefit from emerging AI-driven learning technologies but has yet to fully realize these benefits. The project will directly benefit students participating in the research as they will improve their knowledge estimation skills, prepare more effectively for tests in class, and learn about potential privacy violations and AI biases. Given the fairness focus of the project, the team of researchers will pay special attention to benefits for students from groups traditionally underrepresented in STEM (Science, Technology, Engineering, and Mathematics), ensuring that the AI-powered framework is equally helpful for them and that their perspectives on privacy and fairness receive special attention.This project will advance AI research by incorporating, both, a strict privacy guarantee for student data and fairness considerations across multiple student demographic groups. Additionally, it will advance education research by determining how effective preemptive feedback is for improving knowledge estimation skills, and will examine the mechanism by which preemptively improving knowledge estimation influences academic outcomes. In particular, the project will achieve four research objectives through interdisciplinary innovations in both learning sciences and technology. First, the team will determine how much students' metacognitive calibration can be improved via AI-powered preemptive feedback, which may be perceived differently by students than post hoc feedback. Second, the project will expand theoretical understanding of metacognitive calibration and calibration interventions by studying the mechanism by which the intervention in the project works. Third, the team will address the fundamental tradeoff between the fairness and accuracy of AI models via an innovative federated learning model. Fourth, the team will evaluate the AI framework on real-world education datasets and compare its performance with the state-of-the-art baselines in terms of protecting privacy and mitigating bias. The project team will disseminate results of the project through workshops, publications, and interactive activities, and will train undergraduate and graduate students from diverse backgrounds throughout the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.