Supporting Teacher Practice to Facilitate and Assess Oral Scientific Argumentation: Embedding a Real-Time Assessment of Speaking and Listening into an Argumentation-Rich Curriculum

这是一项早期设计和开发合作研究，提交给探索研究PreK-12 （DRK-12）项目的评估链，响应项目征求NSF 15-592。该项目的根本目的是支持教师围绕口头科学论证的实践和专业学习，以提高课堂实践的质量。为了实现这一目标，该项目将检验一种新的基于技术的课堂论证形成性评估工具的有效性——“诊断小组论证水平”（DiALoG）——为此，心理测量学验证工作已在实验室环境中进行。DiALoG评估工具允许教师记录课堂谈话，并在多个维度（包括个人和人际关系）上显示分数，以进行形成性评估。该项目将与6 -8年级的科学教师合作，通过DiALoG工具生成的实时形成性评估数据来监控和支持论证。DiALoG将与Lawrence Hall of Science开发的“Amplify Science”课程以及一套新开发的“响应式迷你课程”（Responsive Mini-Lessons, RMLs）结合使用，该课程结合了从证据中进行论证的科学实践，该课程由20-30分钟的教学策略组成，旨在帮助教师对学生的思维提供反馈，并对DiALoG工具确定需要进一步支持的论证章节进行后续跟进。这项研究将使对话得以完善和扩大，并评价其对教师教学内容、知识和课堂广泛使用的形成性评价实践的影响。该项目将解决两个具体的研究问题：(1)如何将DiALoG改进为一种在中学课堂上可靠、实用和有用的口头论证形成性评估工具？(2)在实施旨在支持口头论证的科学单元时，DiALoG的使用如何影响教师围绕循证论证的形成性评估实践？为了回答这些问题，该项目将对100名教师进行随机对照试验：50名教师将使用DiALoG教授以论证为重点的课程，50名教师将不使用DiALoG教授相同的课程。控制组和治疗组教师都将收到教授三个Amplify Science课程单元所需的所有数字和物理材料。治疗教师还将获得最新版本的DiALoG，包括相关的rml，以及在Amplify课程中使用DiALoG的支持材料。重点教师小组（5名来自治疗组，5名来自对照组）将成为额外数据收集和分析的主题。将从三个Amplify Science课程单元中选择三个重点课程，让学生进行小组或全班口头辩论。随机对照试验的教师措施将包括经过验证的工具，例如(a)教师教学内容知识的前后评估；(b)课后和单元后调查，让教师自我报告他们的形成性评估方法；(c)重点教室中选定课程的录像。为了观察治疗组和对照组在形成性评价实践方面的潜在差异，将使用协议来分析重点教室的录像，包括(a)改革后的教学观察协议；(b)评估课堂内的科学论证；(c)教师和学生的形成性评估。这项工作的关键成果将是一个基于研究和实地测试的原型，以提高在不同学习环境中可用的中学科学课堂的教学和学习论证质量。

This is an early-stage design and development collaborative study submitted to the assessment strand of the Discovery Research PreK-12 (DRK-12) program, in response to Program Solicitation NSF 15-592. The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. To achieve this purpose, the project will examine the validity of a new technology-based formative assessment tool for classroom argumentation--"Diagnosing the Argumentation Levels of Groups" (DiALoG)--for which psychometric validation work has been conducted in a laboratory setting. The DiALoG assessment tool allows teachers to document classroom talk and display scores across multiple dimensions--both intrapersonal and interpersonal--for formative assessment purposes. The project will work with 6th-8th grade science teachers to monitor and support argumentation through real-time formative assessment data generated by the DiALoG instrument. DiALoG will be used in conjunction with "Amplify Science", a Lawrence Hall of Science-developed curriculum that incorporates the science practice of engaging in argument from evidence, and a suite of newly developed Responsive Mini-Lessons (RMLs), which consist of 20-30 minute instructional strategies designed to assist teachers to provide feedback to students' thinking and follow-up to argumentation episodes that the DiALoG tool identifies in need of further support. The study will allow the refinement and expansion of DiALoG and evaluation of its impact on teacher pedagogical content knowledge and formative assessment practices in widespread classroom use.The project will address two specific research questions: (1) How can DiALoG be refined to provide a formative assessment tool for oral argumentation that is reliable, practical, and useful in middle school classrooms?; and (2) How does the use of DiALoG affect teacher formative assessment practices around evidence-based argumentation, when implementing science units designed to support oral argumentation? In order to answer these questions, the project will conduct a randomized control trial with 100 teachers: 50 will teach argumentation-focused curriculum with DiALoG, 50 will teach the same curriculum without DiALoG. Both control and treatment teachers will receive all digital and physical materials needed to teach three Amplify Science curriculum units. Treatment teachers will be provided also with the most recent version of DiALoG, including the linked RMLs, as well as support materials for using DiALoG with the Amplify curriculum. A subgroup of focus teachers (5 from the treatment group, and 5 from the control group) will be the subject of additional data collection and analysis. Three focus lessons, in which students are engaging in small-group or whole-class oral argumentation, will be selected from each of the three Amplify Science curricular units. Teacher measures for the randomized control trial will include validated instruments, such as (a) a pre- and post-assessment of teacher pedagogical content knowledge; (b) post-lesson and post-unit surveys in which teachers will self-report on their formative assessment practices; and (c) video recordings of selected lessons in the focus classrooms. In order to observe potential differences in formative assessment practices between treatment and control, protocols will be used to analyze the video recordings of focus classrooms, including (a) Reformed Teaching Observation Protocol; (b) Assessment of Scientific Argumentation inside the Classroom; and (c) Formative Assessment for Teachers and Students. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.