FairFL-MC: A Metacognitive Calibration Intervention Powered by Fair and Private Machine Learning

FairFL-MC：由公平和私人机器学习支持的元认知校准干预

• 批准号: 2202481
• 负责人: Dong Wang
• 金额: $ 85万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-01 至 2025-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2202481&HistoricalAwards=false
• 关键词: FairFL; MC; Metacognitive; Calibration; Intervention

Students often have difficulty estimating their own level of knowledge. The goal of this project is to research ways to improve students' ability to estimate their knowledge, using a student support system consisting of short training exercises that will be personalized with artificial intelligence (AI) methods. While there is abundant research on AI methods in educational contexts, such projects rarely consider some of the key social and human factors, such as privacy and fairness, that are needed for widespread adoption of personalized educational software. This project addresses these issues with a novel decentralized AI framework that is specifically for education contexts. The project framework will enable researchers to create AI systems that provide feedback to students as part of their training exercises, all without directly accessing their data and while also training the AI system to reduce biases related to key aspects of students' identity, such as their demographics. The training exercises will include educational activities where students estimate their test scores, receive feedback from the AI system, and reflect on their knowledge. The privacy and fairness capabilities of the project framework will transform postsecondary online learning, which is poised to benefit from emerging AI-driven learning technologies but has yet to fully realize these benefits. The project will directly benefit students participating in the research as they will improve their knowledge estimation skills, prepare more effectively for tests in class, and learn about potential privacy violations and AI biases. Given the fairness focus of the project, the team of researchers will pay special attention to benefits for students from groups traditionally underrepresented in STEM (Science, Technology, Engineering, and Mathematics), ensuring that the AI-powered framework is equally helpful for them and that their perspectives on privacy and fairness receive special attention.This project will advance AI research by incorporating, both, a strict privacy guarantee for student data and fairness considerations across multiple student demographic groups. Additionally, it will advance education research by determining how effective preemptive feedback is for improving knowledge estimation skills, and will examine the mechanism by which preemptively improving knowledge estimation influences academic outcomes. In particular, the project will achieve four research objectives through interdisciplinary innovations in both learning sciences and technology. First, the team will determine how much students' metacognitive calibration can be improved via AI-powered preemptive feedback, which may be perceived differently by students than post hoc feedback. Second, the project will expand theoretical understanding of metacognitive calibration and calibration interventions by studying the mechanism by which the intervention in the project works. Third, the team will address the fundamental tradeoff between the fairness and accuracy of AI models via an innovative federated learning model. Fourth, the team will evaluate the AI framework on real-world education datasets and compare its performance with the state-of-the-art baselines in terms of protecting privacy and mitigating bias. The project team will disseminate results of the project through workshops, publications, and interactive activities, and will train undergraduate and graduate students from diverse backgrounds throughout the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

学生往往难以估计自己的知识水平。该项目的目标是研究如何提高学生的能力，估计他们的知识，使用一个学生支持系统，包括短期的训练练习，将与人工智能（AI）方法个性化。虽然在教育背景下对人工智能方法进行了大量研究，但这些项目很少考虑一些关键的社会和人为因素，例如隐私和公平，而这些因素是广泛采用个性化教育软件所必需的。该项目通过一个专门针对教育环境的新型去中心化AI框架来解决这些问题。该项目框架将使研究人员能够创建人工智能系统，作为训练练习的一部分向学生提供反馈，所有这些都不直接访问他们的数据，同时还训练人工智能系统以减少与学生身份关键方面相关的偏见，例如他们的人口统计数据。培训活动将包括教育活动，学生可以评估他们的考试成绩，从人工智能系统获得反馈，并反思他们的知识。该项目框架的隐私和公平能力将改变中学后在线学习，这将受益于新兴的人工智能驱动的学习技术，但尚未完全实现这些好处。该项目将使参与研究的学生直接受益，因为他们将提高知识评估技能，更有效地准备课堂测试，并了解潜在的隐私侵犯和人工智能偏见。考虑到该项目的公平性，研究人员团队将特别关注传统上在STEM中代表性不足的群体的学生的利益（科学、技术、工程和数学），确保人工智能驱动的框架对他们同样有帮助，并确保他们对隐私和公平的观点得到特别关注。该项目将通过结合，学生数据的严格隐私保证以及对多个学生群体的公平考虑。此外，它将通过确定如何有效的先发制人的反馈是提高知识估计技能，并将检查先发制人的提高知识估计影响学术成果的机制，推进教育研究。特别是，该项目将通过学习科学和技术的跨学科创新实现四个研究目标。首先，该团队将确定学生的元认知校准可以通过人工智能驱动的抢先反馈来改善多少，学生可能会对这种反馈的感知与事后反馈不同。其次，该项目将通过研究项目中干预的机制来扩展对元认知校准和校准干预的理论理解。第三，该团队将通过创新的联邦学习模型来解决AI模型的公平性和准确性之间的根本权衡。第四，该团队将在真实世界的教育数据集上评估人工智能框架，并将其在保护隐私和减轻偏见方面的表现与最先进的基线进行比较。项目团队将通过研讨会、出版物和互动活动传播项目成果，并将在整个项目过程中培训来自不同背景的本科生和研究生。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。