Collaborative Research: STEM Learning Embedded in a Machine-in-the-Loop Collaborative Story Writing Game

协作研究：嵌入机器在环协作故事写作游戏中的 STEM 学习

• 批准号: 2202506
• 负责人: Mohit Iyyer
• 金额: $ 62.14万
• 依托单位: University of Massachusetts Amherst
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2202506&HistoricalAwards=false
• 关键词: Collaborative; Research; STEM; Learning; Embedded

Developing “21st century skills” such as collaboration, communication, critical thinking, and creativity (the 4Cs) has become increasingly important for students to keep up with the ever-evolving labor market of the future. Teaching the 4Cs effectively and efficiently requires deeply intertwining them with core content knowledge areas, since the acquisition of domain knowledge can bolster students’ development of these soft skills. In this project, the investigators take a step towards combining 4C skill development with STEM education by developing a collaborative writing game in which multiple students work together to craft a narrative around embedded STEM education elements. As a key innovation, the investigators will embed this collaborative writing game with natural language processing and artificial intelligence (AI)-based tools to automate fact-checking, feedback, knowledge tracing, and narrative story arc suggestions, which will facilitate students’ progress toward mastery while reducing teacher workload. Overall, this project has the potential to increase student engagement in STEM learning activities and improve learning outcomes. The project will be grounded in StoriumEdu, a collaborative story writing platform, therefore directly benefiting its user base of 2,000 K-12 classrooms with over 27,000 students and potentially an even larger number of students through the dissemination of the team’s research findings. This major technical goals of this project are intended to augment scientific writing instruction with AI-based tools. To achieve these goals, the project will develop novel technologies that automatically provide writing assistance and feedback, and these tools will be deployed into K-12 classrooms via the StoriumEdu platform in order to evaluate their effectiveness. A core technical challenge is to assess the factuality of student writing by building machine learning models for fact-checking. The team proposes to design retrieval-augmented neural networks that can localize spans within student-written text that exhibit scientific misunderstandings. These spans will then be connected with relevant passages from textbooks or online articles to enable students to easily correct their errors. After developing fact-checking methods, the team will also focus on knowledge tracing, which allows measuring student progress over time in terms of which concepts they have mastered or are still struggling with. The knowledge tracing models will be developed with feedback from scientific literacy experts. The output of these models informs the final aspect of this project, which aims to generate narrative progressions associated with conceptual misunderstandings. This will allow students to engage more strongly with concepts that they have yet to master, which maximizes the writing platform’s pedagogical potential. Taken as a whole, this project’s research contributions synthesize novel NLP methods with educational progress tracking and feedback systems in an effort to improve STEM learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

发展“21世纪世纪技能”，如协作，沟通，批判性思维和创造力（4C）已成为学生跟上不断发展的未来劳动力市场越来越重要。有效和高效地教授4C需要将它们与核心内容知识领域深深交织在一起，因为领域知识的获得可以促进学生对这些软技能的发展。在这个项目中，研究人员通过开发一个协作写作游戏，将4C技能开发与STEM教育相结合，在这个游戏中，多名学生共同努力，围绕嵌入式STEM教育元素制作一个叙事。作为一项关键的创新，研究人员将把这个协作写作游戏与自然语言处理和基于人工智能（AI）的工具结合起来，自动进行事实检查、反馈、知识追踪和叙事故事弧建议，这将促进学生的掌握进度，同时减少教师的工作量。总体而言，该项目有可能增加学生参与STEM学习活动并提高学习成果。该项目将以StoriumEdu为基础，StoriumEdu是一个协作故事写作平台，因此直接受益于其2，000个K-12教室的用户群，其中有27，000多名学生，并可能通过传播团队的研究成果而受益于更多的学生。该项目的主要技术目标是通过基于AI的工具来增强科学写作教学。为了实现这些目标，该项目将开发自动提供写作帮助和反馈的新技术，这些工具将通过StoriumEdu平台部署到K-12教室，以评估其有效性。一个核心的技术挑战是通过构建用于事实检查的机器学习模型来评估学生写作的真实性。该团队建议设计检索增强神经网络，可以定位学生写作文本中表现出科学误解的跨度。然后，这些跨度将与教科书或在线文章中的相关段落相连接，使学生能够轻松纠正错误。在开发了事实核查方法之后，该团队还将专注于知识追踪，这可以衡量学生随着时间的推移所取得的进展，即他们已经掌握了哪些概念或仍在努力学习哪些概念。将根据科学素养专家的反馈意见制定知识追踪模型。这些模型的输出为该项目的最后一个方面提供了信息，该项目旨在产生与概念误解相关的叙事进展。这将使学生能够更强烈地参与他们尚未掌握的概念，从而最大限度地发挥写作平台的教学潜力。作为一个整体，该项目的研究成果将创新的NLP方法与教育进展跟踪和反馈系统相结合，以提高STEM学习。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization

• DOI: 10.48550/arxiv.2301.13298
• 发表时间: 2023-01
• 期刊: ArXiv
• 作者: Kalpesh Krishna;Erin Bransom;Bailey Kuehl;Mohit Iyyer;Pradeep Dasigi;Arman Cohan;Kyle Lo

Open-ended Knowledge Tracing for Computer Science Education

• DOI: 10.18653/v1/2022.emnlp-main.254
• 发表时间: 2022
• 作者: Naiming Liu;Zichao Wang

A Critical Evaluation of Evaluations for Long-form Question Answering

对长式问答评估的批判性评估

• DOI: 10.18653/v1/2023.acl-long.181
• 发表时间: 2023
• 期刊: Association for Computational Linguistics
• 作者: Xu, Fangyuan;Song, Yixiao;Iyyer, Mohit;Choi, Eunsol
• 通讯作者: Choi, Eunsol