Student Reasoning Patterns in Next Generation Science Standards Assessment

该项目由教育测试服务中心的一个团队领导，其目标是开发自动化工具，通过这些工具可以对符合下一代科学标准（NGSS）的评估进行评分，以揭示学生的推理模式，其中一些模式可能反映出学生推理中的特定弱点。推理模式是指学生在理解自然现象或试图解决问题时的各种思维方式。研究人员将进行概念验证研究，以开发自动诊断方法，根据中学生对生态学概念理解评估的书面回答，识别他们的推理模式。随着各州越来越倾向于实施与NGSS一致的评估，基于个别学生推理模式的反馈将使教师能够开发更个性化的反馈，也将支持基于学生知识和学习方式的自动化教学设计，而不是简单地基于他们是否回答正确。该项目由EHR核心研究（ECR）项目资助，该项目支持推进STEM学习基础研究文献的工作。研究人员将进行概念验证研究，通过调查学生对从ngss一致的评估数据库中收集的构建测试项目的反应，来确定学生理解生态系统的推理模式。在研究的第一阶段，研究人员将使用与NGSS一致的三维（内容知识、程序知识和认知知识）方法进行评估。他们将利用尖端的自然语言处理（NLP）技术来识别学生的推理模式，试图将文本标记为数据描述和系统关系描述，并试图识别与集成推理相反的表面集成。调查人员将参与一个迭代过程，将自动化工具产生的分类与人工评分者产生的分类进行比较。随着这种分类的发展和验证，研究人员将证明稍后尝试开发基于nlp的自动化系统的可行性，该系统可以向学生提供即时反馈，识别推理中的弱点，而不仅仅是答案是否正确，并向教师提供反馈，使他们能够为个别学生量身定制教学。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this project, led by a team at Educational Testing Services, is to develop automated tools by which assessments aligned with the Next Generation Science Standards (NGSS) can be scored to reveal student reasoning patterns, some of which would reflect particular weaknesses in student reasoning. Reasoning patterns refer to various ways of student thinking when making sense of a natural phenomenon or trying to solve a problem. The investigators will conduct a proof of concept study to develop automated diagnostics that can identify middle-school students’ reasoning patterns based on their written responses to assessments of their understanding of concepts in ecology. With the states increasingly moving towards implementing assessments aligned to the NGSS, feedback based on individual students’ reasoning patterns would allow teachers the ability to develop more individualized feedback and would also support the design of automated instruction based on evidence of what students know and how they learn, rather than instruction based simply on whether they had answered correctly or not. The project is funded by the EHR Core Research (ECR) program, which supports work that advances the fundamental research literature on STEM learning. The investigators will conduct a proof of concept study to identity student reasoning patterns for making sense of ecosystems by investigating student responses to constructed test items collected from an NGSS-aligned assessment database. In the first stage of the study, the investigators will use a 3-dimensional (content knowledge, procedural knowledge, and epistemic knowledge) approach to assessment that aligns with the NGSS. They will leverage cutting-edge natural language processing (NLP) techniques to identify student reasoning patterns, attempting to label text as data description and system relationship description and attempting to identify superficial integration as opposed to integrated reasoning. The investigators will engage in an iterative process to compare the classification produced by the automated tools with that produced by human scorers. With this classification developed and validated, the investigators will have demonstrated the feasibility of later attempting to develop an NLP-based automated system that could provide immediate feedback to students, identifying weaknesses in reasoning rather than only whether an answer was correct, and to teachers, allowing them to tailor instruction to individual students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.