FairFL-MC: A Metacognitive Calibration Intervention Powered by Fair and Private Machine Learning

学生们常常难以估计自己的知识水平。该项目的目标是研究如何提高学生估计自己知识的能力，使用由人工智能（AI）方法个性化的短训练练习组成的学生支持系统。虽然在教育环境中有大量关于人工智能方法的研究，但这些项目很少考虑到一些关键的社会和人为因素，比如隐私和公平，而这些因素是广泛采用个性化教育软件所需要的。这个项目用一个新颖的去中心化的人工智能框架解决了这些问题，这个框架是专门为教育环境设计的。该项目框架将使研究人员能够创建人工智能系统，为学生提供反馈，作为他们训练练习的一部分，所有这些都不需要直接访问他们的数据，同时还可以训练人工智能系统减少与学生身份的关键方面相关的偏见，例如人口统计数据。训练演习将包括教育活动，学生们可以评估自己的考试成绩，接收人工智能系统的反馈，并反思自己的知识。该项目框架的隐私和公平能力将改变高等教育在线学习，后者有望从新兴的人工智能驱动的学习技术中受益，但尚未充分实现这些好处。该项目将直接使参与研究的学生受益，因为他们将提高他们的知识估计技能，更有效地为课堂考试做准备，并了解潜在的隐私侵犯和人工智能偏见。鉴于该项目的公平性重点，研究团队将特别关注STEM（科学、技术、工程和数学）中传统上代表性不足的群体的学生的利益，确保人工智能支持的框架对他们同样有帮助，并确保他们对隐私和公平性的看法得到特别关注。该项目将通过结合对学生数据的严格隐私保证和对多个学生群体的公平考虑来推进人工智能研究。此外，它将通过确定先发制人的反馈对提高知识估计技能的有效性来推进教育研究，并将研究先发制人地提高知识估计影响学术成果的机制。特别是，该项目将通过学习科学和技术的跨学科创新来实现四个研究目标。首先，该团队将确定通过人工智能驱动的先发制人的反馈，学生的元认知校准可以得到多大程度的改善，这可能与事后反馈的感受不同。其次，本项目将通过研究元认知校准和校准干预在项目中的作用机制，拓展对元认知校准和校准干预的理论认识。第三，该团队将通过创新的联邦学习模型解决人工智能模型的公平性和准确性之间的基本权衡。第四，该团队将在现实世界的教育数据集上评估人工智能框架，并将其在保护隐私和减轻偏见方面的表现与最先进的基线进行比较。项目团队将通过研讨会、出版物和互动活动传播项目成果，并在整个项目过程中培训来自不同背景的本科生和研究生。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Students often have difficulty estimating their own level of knowledge. The goal of this project is to research ways to improve students' ability to estimate their knowledge, using a student support system consisting of short training exercises that will be personalized with artificial intelligence (AI) methods. While there is abundant research on AI methods in educational contexts, such projects rarely consider some of the key social and human factors, such as privacy and fairness, that are needed for widespread adoption of personalized educational software. This project addresses these issues with a novel decentralized AI framework that is specifically for education contexts. The project framework will enable researchers to create AI systems that provide feedback to students as part of their training exercises, all without directly accessing their data and while also training the AI system to reduce biases related to key aspects of students' identity, such as their demographics. The training exercises will include educational activities where students estimate their test scores, receive feedback from the AI system, and reflect on their knowledge. The privacy and fairness capabilities of the project framework will transform postsecondary online learning, which is poised to benefit from emerging AI-driven learning technologies but has yet to fully realize these benefits. The project will directly benefit students participating in the research as they will improve their knowledge estimation skills, prepare more effectively for tests in class, and learn about potential privacy violations and AI biases. Given the fairness focus of the project, the team of researchers will pay special attention to benefits for students from groups traditionally underrepresented in STEM (Science, Technology, Engineering, and Mathematics), ensuring that the AI-powered framework is equally helpful for them and that their perspectives on privacy and fairness receive special attention.This project will advance AI research by incorporating, both, a strict privacy guarantee for student data and fairness considerations across multiple student demographic groups. Additionally, it will advance education research by determining how effective preemptive feedback is for improving knowledge estimation skills, and will examine the mechanism by which preemptively improving knowledge estimation influences academic outcomes. In particular, the project will achieve four research objectives through interdisciplinary innovations in both learning sciences and technology. First, the team will determine how much students' metacognitive calibration can be improved via AI-powered preemptive feedback, which may be perceived differently by students than post hoc feedback. Second, the project will expand theoretical understanding of metacognitive calibration and calibration interventions by studying the mechanism by which the intervention in the project works. Third, the team will address the fundamental tradeoff between the fairness and accuracy of AI models via an innovative federated learning model. Fourth, the team will evaluate the AI framework on real-world education datasets and compare its performance with the state-of-the-art baselines in terms of protecting privacy and mitigating bias. The project team will disseminate results of the project through workshops, publications, and interactive activities, and will train undergraduate and graduate students from diverse backgrounds throughout the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.