Evaluating Effects of Automatic Feedback Aligned to a Learning Progression to Promote Knowledge-In-Use

该项目考察了一个评估系统的效果，该系统根据学生在化学和物理方面的开放式评估反应自动生成反馈，该反馈与先前开发的学习进度一致，该进度描述了学生对电相互作用可以发展的更复杂的理解。研究人员将设计和测试一个使用机器学习的自动评估评分系统。评分系统将为学生提供个性化的反馈，并向老师提供课堂总结。然后，这可以作为形成性评估，以匹配现有的高中物理科学课程，旨在满足下一代科学标准的表现期望。然后，该项目将检查自动反馈是否支持学生的学习成果和他们在电交互学习进展方面的发展。该项目通过让学生参与科学实践（如建模），利用关键学科思想和使用横切概念来理解引人注目的现象，以及向学生提供实时反馈，来促进学生的科学知识。深化学生的科学知识要求他们有机会解决结构不良、复杂的问题，并建立现实世界现象的模型。该项目通过研究如何通过对学生表现的及时和富有成效的反馈来评估和支持学生对这些科学问题的反应，从而符合国家利益。该项目有两个研究问题：1)在与下一代科学标准相一致的物理科学学习进程中，自动反馈对学生表现的影响是什么？2)自动反馈对学生在学习进阶中如何将想法与进步联系起来有什么影响？为了解决这些问题，该项目使用了与下一代科学标准相一致的课程，并为学生学习电子相互作用如何使材料粘在一起或被排斥的有效学习过程。该项目有一个自动评分工具，它应用自然语言处理、图像识别和监督机器学习来为学生的解释和建模反应打分。该项目将评分工具嵌入到新设计和开发的门户网站中，以便将课程材料、评估和自动评分与实时反馈相结合。学生对课程中嵌入的形成性评估项目做出反应；其中的一个子集由机器学习算法自动评分。根据计算机对学生的回答进行分类，学生会收到自动反馈。学生的学习，根据学习进展水平的进步来定义，是用项目反应理论来衡量学生在课程单元开始和结束时给出的总结性评估中的表现。该项目还使用网络和聚类分析来分析学生对形成性项目的反应，以确定学生对反馈的反应方式，并在学习过程中取得进展。该项目的成果将指导如何就科学成绩评估项目向学生提供适当的反馈，并提供基于机器学习的评分模型表示和解释工具。该项目将完善由学习材料、活动和评估项目组成的高中物理科学在线课程，并产生相关的自动评分过程和个性化反馈。探索研究preK-12项目（DRK-12）旨在通过研究和开发创新资源、模型和工具，显著提高preK-12学生和教师对科学、技术、工程和数学（STEM）的学习和教学。DRK-12计划中的项目建立在STEM教育的基础研究和先前的研究和开发工作的基础上，为拟议的项目提供了理论和实证依据。 该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project examines the effect of an assessment system that automatically generates feedback based on students’ open-ended assessment responses in chemistry and physics consistent with a previously-developed learning progression that describes the successively more complex understandings students can develop about electrical interactions. The researchers will design and test an automated assessment scoring system using machine learning. The scoring system will provide individualized feedback to students and class summaries to their teachers. This could then serve as a formative assessment to match an existing high school physical science curriculum designed to meet performance expectations in the Next Generation Science Standards. The project will then examine whether the automatic feedback supports students’ learning outcomes and their development with respect to the learning progression on electrical interactions. The project promotes students’ knowledge of science by engaging them in scientific practices, like modeling, with key disciplinary ideas and using crosscutting concepts to make sense of compelling phenomena and by providing real-time feedback to students. Deepening students’ knowledge of science requires that they have opportunities to solve ill-structured, complex problems and to create models of real-world phenomena. This project serves the national interest by examining how to assess and support students in responding to such problems in science through timely and productive feedback about their performances.The project has two research questions: 1) What is the effect of automatic feedback on student performance along a previously validated learning progression for physical science aligned with the Next Generation Science Standards? 2) What is the effect of automatic feedback on how students connect ideas to advance in learning progression levels? To address these questions the project uses a curriculum aligned with the Next Generation Science Standards and a validated learning progression for student learning about how electrical interactions allow materials to stick together or be repelled. The project has an automatic scoring tool that applies natural language processing, image recognition, and supervised machine learning to score students’ explanations and modeling responses. The project embeds the scoring tool in a newly designed and developed web-portal to allow the integration of curriculum materials, assessments, and automatic scoring with real-time feedback. Students respond to formative assessment items embedded in the curriculum; a subset of these are automatically scored by machine learning algorithms. Students receive automatic feedback on their responses based on the computer classification of their responses. Student learning, as defined by advancement on learning progression levels, is measured using item response theory on student performance on the summative assessments given at the start and end of the curriculum units. The project also uses network and cluster analyses for student responses to formative items to identify ways that students respond to feedback and advance along the learning progression. Outcomes from this project will provide guidance on how to provide appropriate feedback to students on science performance assessment items and a machine learning based tool for scoring model representations and explanations. The project will refine an online curriculum for high school physical science composed of learning materials, activities and assessment items, and produce an associated automatic scoring process and individualized feedback. The Discovery Research preK-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.