Evaluating Effects of Automatic Feedback Aligned to a Learning Progression to Promote Knowledge-In-Use

该项目研究了一个评估系统的效果，该系统根据学生在化学和物理方面的开放式评估反应自动生成反馈，与先前开发的学习进展一致，该学习进展描述了学生可以开发的关于电相互作用的连续更复杂的理解。研究人员将使用机器学习设计和测试一个自动评估评分系统。评分系统将提供个性化的反馈给学生和课堂总结给他们的老师。然后，这可以作为一个形成性评估，以匹配现有的高中物理科学课程，旨在满足下一代科学标准的性能期望。本研究将探讨自动反馈是否支持学生的学习成果，以及他们在电互动学习进展方面的发展。该项目通过让学生参与科学实践，如建模，关键学科思想和使用横切概念来理解引人注目的现象，并通过向学生提供实时反馈，促进学生的科学知识。加深学生的科学知识需要他们有机会解决结构不良，复杂的问题，并创建现实世界的现象模型。本项目旨在通过对学生的表现进行及时有效的反馈来评估和支持学生应对科学中的此类问题，从而为国家利益服务。本项目有两个研究问题：1）自动反馈对学生表现的影响沿着先前验证的符合下一代科学标准的物理科学学习进展？2)自动反馈对学生如何将想法联系起来以提高学习进度水平有什么影响？为了解决这些问题，该项目使用了与下一代科学标准相一致的课程，以及经过验证的学习进度，让学生学习电气相互作用如何使材料粘在一起或被排斥。该项目有一个自动评分工具，应用自然语言处理，图像识别和监督机器学习来评分学生的解释和建模反应。该项目将评分工具嵌入一个新设计和开发的门户网站，以整合课程材料、评估和自动评分与实时反馈。学生对课程中嵌入的形成性评估项目做出反应;其中一部分由机器学习算法自动评分。学生会根据他们的回答的计算机分类收到自动反馈。学生的学习，如在学习进展水平的进步所定义的，是衡量使用项目反应理论对学生的表现在总结性评估给予在课程单元的开始和结束。该项目还使用网络和聚类分析学生的反应形成的项目，以确定学生的反馈和推进沿着学习进展的方式。该项目的成果将指导如何向学生提供科学表现评估项目的适当反馈，以及基于机器学习的工具，用于评分模型表示和解释。该项目将完善高中物理科学的在线课程，包括学习材料，活动和评估项目，并产生相关的自动评分过程和个性化反馈。探索研究preK-12计划（DRK-12）旨在通过研究和开发创新资源，模型和工具，显着提高preK-12学生和教师的科学，技术，工程和数学（STEM）的学习和教学。DRK-12项目中的项目建立在STEM教育的基础研究以及为拟议项目提供理论和经验依据的先前研究和开发工作的基础上。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project examines the effect of an assessment system that automatically generates feedback based on students’ open-ended assessment responses in chemistry and physics consistent with a previously-developed learning progression that describes the successively more complex understandings students can develop about electrical interactions. The researchers will design and test an automated assessment scoring system using machine learning. The scoring system will provide individualized feedback to students and class summaries to their teachers. This could then serve as a formative assessment to match an existing high school physical science curriculum designed to meet performance expectations in the Next Generation Science Standards. The project will then examine whether the automatic feedback supports students’ learning outcomes and their development with respect to the learning progression on electrical interactions. The project promotes students’ knowledge of science by engaging them in scientific practices, like modeling, with key disciplinary ideas and using crosscutting concepts to make sense of compelling phenomena and by providing real-time feedback to students. Deepening students’ knowledge of science requires that they have opportunities to solve ill-structured, complex problems and to create models of real-world phenomena. This project serves the national interest by examining how to assess and support students in responding to such problems in science through timely and productive feedback about their performances.The project has two research questions: 1) What is the effect of automatic feedback on student performance along a previously validated learning progression for physical science aligned with the Next Generation Science Standards? 2) What is the effect of automatic feedback on how students connect ideas to advance in learning progression levels? To address these questions the project uses a curriculum aligned with the Next Generation Science Standards and a validated learning progression for student learning about how electrical interactions allow materials to stick together or be repelled. The project has an automatic scoring tool that applies natural language processing, image recognition, and supervised machine learning to score students’ explanations and modeling responses. The project embeds the scoring tool in a newly designed and developed web-portal to allow the integration of curriculum materials, assessments, and automatic scoring with real-time feedback. Students respond to formative assessment items embedded in the curriculum; a subset of these are automatically scored by machine learning algorithms. Students receive automatic feedback on their responses based on the computer classification of their responses. Student learning, as defined by advancement on learning progression levels, is measured using item response theory on student performance on the summative assessments given at the start and end of the curriculum units. The project also uses network and cluster analyses for student responses to formative items to identify ways that students respond to feedback and advance along the learning progression. Outcomes from this project will provide guidance on how to provide appropriate feedback to students on science performance assessment items and a machine learning based tool for scoring model representations and explanations. The project will refine an online curriculum for high school physical science composed of learning materials, activities and assessment items, and produce an associated automatic scoring process and individualized feedback. The Discovery Research preK-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.