权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CAREER: Generative Item, Response, and Feedback Models in Assessment and Learning

职业：评估和学习中的生成项目、响应和反馈模型

基本信息

批准号：
2237676
负责人：
Shiting Lan
金额：
$ 64.46万
依托单位：
University of Massachusetts Amherst
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2023
资助国家：
美国
起止时间：
2023-05-15 至 2028-04-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2237676&HistoricalAwards=false
关键词：
CAREER Generative Item Response Feedback

项目摘要

Personalized tutoring and feedback on performance or knowledge mastery, are two instructional strategies that have been shown to be effective at improving student learning outcomes. However, implementing these strategies, especially at scale, is costly in terms of the human resources required to provide them effectively. The use of Artificial Intelligence (AI) to provide students with feedback and personalized tutoring in digital learning platforms has the potential to reduce the human capital required to provide these services and to service growing numbers of learners effectively. This CAREER project will leverage generative language models (GLMs), a recent innovation in AI machine learning, to estimate learner knowledge levels and identify specific errors from open-ended learner responses. The resulting system will then be able to automatically generate personalized items and feedback, to support teachers and learners. Primarily grounded in middle-school math education with data collection and evaluation supported by ASSISTments and OpenStax, this CAREER project has the potential to benefit many teachers and learners. Other potential outcomes of his CAREER project include activities that expand the access of minority learners to real-world applications of AI and a new course on AI for education. This CAREER project includes three major research threads. First, the project team will develop a family of open-ended item response theory and knowledge tracing frameworks for open-ended math items. The key technical challenge will be to inject learner knowledge states to steer GLMs towards generating personalized response predictions according to each learner’s knowledge on different skills. These models will power teacher dashboard tools and learner error detection tools during tutoring activities. Second, the project team will develop GLM-based automated math item generation methods to meet the needs and interests of each learner and evaluate them in a randomized controlled trial. The key technical challenge will be to control the generated items according to human specifications on item context and both mathematical and language complexity. Third, the project team will develop a GLM-based automated feedback generation framework and explore its usage in both common wrong answer feedback and tutoring dialogue turn generation. The key technical challenge will be to learn how to leverage effective teacher-written feedback messages and use them as input examples for GLMs. The team will also explore learning-from-teacher-edit methods to constantly improve the quality of generated feedback over time.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

个性化辅导和对表现或知识掌握的反馈是两种已被证明能有效提高学生学习效果的教学策略。然而，实施这些战略，特别是大规模实施这些战略，就有效提供这些战略所需的人力资源而言，代价高昂。使用人工智能（AI）在数字学习平台上为学生提供反馈和个性化辅导，有可能减少提供这些服务所需的人力资本，并有效地为越来越多的学习者提供服务。这个CAREER项目将利用人工智能机器学习的最新创新--生成语言模型（GLM）来估计学习者的知识水平，并从开放式学习者的反应中识别特定的错误。由此产生的系统将能够自动生成个性化的项目和反馈，以支持教师和学习者。该项目以中学数学教育为基础，由ASSISTments和OpenStax支持数据收集和评估，有可能使许多教师和学习者受益。他的CAREER项目的其他潜在成果包括扩大少数民族学习者对人工智能现实世界应用的访问的活动，以及关于人工智能教育的新课程。这个职业生涯项目包括三个主要的研究线索。首先，项目团队将为开放式数学项目开发一系列开放式项目反应理论和知识追踪框架。关键的技术挑战将是注入学习者的知识状态，以引导GLM根据每个学习者对不同技能的知识生成个性化的响应预测。这些模型将在辅导活动中为教师仪表板工具和学习者错误检测工具提供动力。其次，项目团队将开发基于GLM的自动数学题生成方法，以满足每个学习者的需求和兴趣，并在随机对照试验中对其进行评估。关键的技术挑战将是根据人类对项目上下文的规范以及数学和语言复杂性来控制生成的项目。第三，项目团队将开发一个基于GLM的自动反馈生成框架，并探索其在常见错误答案反馈和辅导对话轮生成中的使用。关键的技术挑战将是学习如何利用有效的教师书面反馈信息，并将其作为GLM的输入示例。这个奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（8）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Automated Distractor and Feedback Generation for Math Multiple-choice Questions via In-context Learning

DOI：
发表时间：
2023-08
期刊：
影响因子：
0
作者：
Hunter McNichols;Wanyong Feng;Jaewook Lee;Alexander Scarlatos;Digory Smith;Simon Woodhead;Andrew S. Lan
通讯作者：
Hunter McNichols;Wanyong Feng;Jaewook Lee;Alexander Scarlatos;Digory Smith;Simon Woodhead;Andrew S. Lan

Balancing Test Accuracy and Security in Computerized Adaptive Testing

平衡计算机化自适应测试中的测试准确性和安全性