Collaborative Research: Development of Natural Language Processing Techniques to Improve Students' Revision of Evidence Use in Argument Writing

合作研究：开发自然语言处理技术以提高学生对论证写作中证据使用的修改

• 批准号: 2202347
• 负责人: Diane Litman
• 金额: $ 67.99万
• 依托单位: University of Pittsburgh
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-01 至 2025-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2202347&HistoricalAwards=false
• 关键词: Collaborative; Research; Development; Natural; Language

Writing is foundational to learning in multiple disciplines. It is a critical process by which students make sense of ideas – particularly from source texts – and bring them to bear to demonstrate their emerging understanding of concepts and to make sound arguments. Recognizing the importance of argumentative writing, multiple educational technologies driven by natural language processing (NLP) have been developed to support students and teachers in these processes. However, evidence is modest that such systems improve writing skills, and this is especially the case for younger students. One reason is that NLP technologies have only recently matured to the point that it is possible to provide feedback keyed to the content of students’ writing. A second reason is that many students lack the strategic knowledge and skills needed to revise their essays even after receiving writing feedback. An educational technology that assesses students’ skill at revising their writing and that provides feedback on their revision attempts would support the development of this critical skill, while placing no additional burden on teachers. Such a technology has the potential to prepare a new generation of students for productively writing and revising argumentative essays, a skill they will need in order to be prepared for the educational and workplace settings of the future.To address the limitations of existing educational technologies for writing, the research team will develop a system that leverages NLP to provide students with formative feedback on the quality of their revisions. The team will 1) develop and establish the reliability and validity of new measures of revision quality in response to formative feedback on evidence use, 2) use NLP to automate the scoring of revisions using these measures, 3) provide formative feedback to students based on the automated revision scoring, and 4) evaluate the utility of this feedback in improving student writing and revision in classroom settings. The team hypothesizes that such a system will improve students’ implementation of feedback messages on text-based argument writing, leading toward more successful revision and ultimately more successful writing. For learning researchers and educators, the revision quality measures will provide detailed information about how students implement formative feedback. Few summative or formative assessments currently exist that provide this type of information. For technology researchers, the automated revision scoring will extend prior writing analysis research in novel ways, e.g., by assessing the quality of revisions between essay drafts and by incorporating alignment with prior formative feedback into the assessment. Multiple types of NLP models will be developed to examine tradeoffs between model type and differing evaluation dimensions such as reliability, transparency, and fairness.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

写作是多学科学习的基础。这是一个关键的过程，学生通过这个过程来理解想法--特别是从原始文本中--并使他们承担起展示他们对概念的新兴理解和提出合理论点的能力。认识到议论文写作的重要性，自然语言处理(NLP)驱动的多种教育技术已经被开发出来，以支持学生和教师在这些过程中。然而，有少量证据表明，这种系统可以提高写作技能，对于年纪较小的学生来说尤其如此。一个原因是，NLP技术最近才成熟到可以根据学生写作内容提供反馈的程度。第二个原因是，许多学生即使在收到写作反馈后，也缺乏修改论文所需的战略知识和技能。一种评估学生修改写作技能并对他们的修改尝试提供反馈的教育技术将支持这一关键技能的发展，同时不会给教师带来额外的负担。这种技术有可能使新一代学生为高效写作和修改议论文做好准备，这是他们为未来的教育和工作环境做准备所需要的技能。为了解决现有写作教育技术的局限性，研究小组将开发一种系统，利用NLP为学生提供关于修改质量的形成性反馈。该团队将1)开发和建立针对证据使用的形成性反馈的新的修改质量测量的可靠性和有效性，2)使用NLP来使用这些测量来自动对修改进行评分，3)基于自动修改评分向学生提供形成性反馈，以及4)评估这种反馈在改进学生写作和课堂环境中修改的效用。该团队假设，这样的系统将改善学生对基于文本的论点写作的反馈信息的实施，导致更成功的修改，并最终导致更成功的写作。对于学习研究人员和教育工作者来说，修订质量措施将提供有关学生如何实施形成性反馈的详细信息。目前提供这类信息的总结性或形成性评估很少。对于技术研究人员，自动修改评分将以新颖的方式扩展先前的写作分析研究，例如，通过评估论文草稿之间的修改质量，以及通过将与先前形成性反馈的一致性纳入评估。将开发多种类型的NLP模型，以检查模型类型与不同评估维度(如可靠性、透明度和公正性)之间的权衡。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。