Investigating How to Enhance Scientific Argumentation through Automated Feedback in the Context of Two High School Earth Science Curriculum Units

研究如何在两个高中地球科学课程单元的背景下通过自动反馈增强科学论证

• 批准号: 1418019
• 负责人: Ou Liu
• 金额: $ 249.56万
• 依托单位: Educational Testing Service
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1418019&HistoricalAwards=false
• 关键词: Investigating; How; Enhance; Scientific; Argumentation

With the current emphasis on learning science by actively engaging in the practices of science, and the call for integration of instruction and assessment; new resources, models, and technologies are being developed to improve K-12 science learning. Student assessment has become a nationwide educational priority due, in part, to the need for relevant and timely data that inform teachers, administrators, researchers, and the public about how all students perform and think while learning science. This project responds to the need for technology-enhanced assessments that promote the critical practice of scientific argumentation--making and explaining a claim from evidence about a scientific question and critically evaluating sources of uncertainty in the claim. It will investigate how to enhance this practice through automated scoring and immediate feedback in the context of two high school curriculum units--climate change and fresh-water availability--in schools with diverse student populations. The project will apply advanced automated scoring tools to students' written scientific arguments, provide individual students with customized feedback, and teachers with class-level information to assist them with improving scientific argumentation. The key outcome of this effort will be a technology-supported assessment model of how to advance the understanding of argumentation, and the use of multi-level feedback as a component of effective teaching and learning. The project will strengthen the program's current set of funded activities on assessment, focusing these efforts on students' argumentation as a complex science practice.This design and development research targets high school students (n=1,940) and teachers (n=22) in up to 10 states over four years. The research questions are: (1) To what extent can automated scoring tools, such as c-rater and c-rater-ML, diagnose students' explanations and uncertainty articulations as compared to human diagnosis?; (2) How should feedback be designed and delivered to help students improve scientific argumentation?; (3) How do teachers use and interact with class-level automated scores and feedback to support students' scientific argumentation with real-data and models?; and (4) How do students perceive their overall experience with the automated scores and immediate feedback when learning core ideas in climate change and fresh-water availability topics through scientific argumentation enhanced with modeling? In Years 1 and 2, plans are to conduct feasibility studies to build automated scoring models and design feedback for previously tested assessments for the two curriculum units. In Year 3, the project will implement design studies in order to identify effective feedback through random assignment. In Year 4, a pilot study will investigate if effective feedback should be offered with or without scores. The project will employ a mixed-methods approach. Data-gathering strategies will include classroom observations; screencast and log data of teachers' and students' interaction with automated feedback; teachers' and students' surveys with selected- and open-ended questions; and in-depth interviews with teachers and students. All constructed-response explanations and uncertainty items will be scored using automated scoring engines with fine-grained rubrics. Data analysis strategies will include multiple criteria to evaluate the quality of automated scores; descriptive statistical abalyses; analysis of variance to investigate differences in outcomes from the designed studies' pre/posttests and embedded assessments; analysis of covariance to investigate student learning trajectories; two-level hierarchical linear modeling to study the clustering of students within a class; and analysis of screencasts and log data.

随着当前对通过积极参与科学实践来学习科学的重视，以及对教学和评估一体化的呼吁;正在开发新的资源，模型和技术来改善K-12科学学习。学生评估已经成为全国教育的优先事项，部分原因是需要相关和及时的数据，告知教师，管理人员，研究人员和公众所有学生在学习科学时的表现和思考。该项目是为了满足技术强化评估的需要，以促进科学论证的批判性实践-根据关于科学问题的证据提出和解释一项主张，并批判性地评估该主张中的不确定性来源。它将研究如何在学生人数不同的学校，通过两个高中课程单元-气候变化和淡水供应-的自动评分和即时反馈，加强这一做法。该项目将对学生的书面科学论证应用先进的自动评分工具，为个别学生提供定制的反馈，并为教师提供班级级信息，以帮助他们改进科学论证。这项工作的主要成果将是一个技术支持的评估模型，如何提高对论证的理解，并使用多级反馈作为有效教学和学习的一个组成部分。该项目将加强该计划目前资助的评估活动，将这些努力集中在学生的论证作为一个复杂的科学实践。这项设计和开发研究的目标是高中学生（n= 1，940）和教师（n=22）在四年内在多达10个州。研究问题是：（1）与人类诊断相比，自动评分工具（如c-rater和c-rater-ML）在多大程度上可以诊断学生的解释和不确定性表达？(2)如何设计和提供反馈以帮助学生提高科学论证水平？(3)教师如何使用班级级自动评分和反馈并与之交互，以支持学生用真实数据和模型进行科学论证？以及（4）当学生通过建模增强的科学论证学习气候变化和淡水可用性主题的核心思想时，他们如何看待自动评分和即时反馈的整体体验？在第一年和第二年，计划进行可行性研究，以建立自动评分模型，并为两个课程单元的先前测试评估设计反馈。在第三年，该项目将实施设计研究，以确定通过随机分配有效的反馈。在第四年，一项试点研究将调查是否应该提供有效的反馈，有或没有分数。该项目将采用混合方法。数据收集战略将包括课堂观察;教师和学生互动的屏幕播放和记录数据，并有自动反馈;教师和学生调查，并有选定的和开放式的问题;对教师和学生进行深入访谈。所有构建的回答解释和不确定性项目都将使用自动评分引擎进行评分，并使用细粒度的规则。数据分析策略将包括多个标准，以评估自动评分的质量;描述性统计分析;方差分析，以调查设计的研究的前/后测试和嵌入式评估的结果差异;协方差分析，以调查学生的学习轨迹;两级分层线性模型，以研究班级内学生的聚类;以及屏幕录像和日志数据的分析。