Supporting Teacher Practice to Facilitate and Assess Oral Scientific Argumentation: Embedding a Real-Time Assessment of Speaking and Listening into an Argumentation-Rich Curriculum

这是一项提交给Discovery Research PreK-12（DRK-12）项目评估链的早期设计和开发合作研究，以响应项目征集NSF 15-592。该项目的基本目的是支持教师实践和专业学习围绕口头科学论证，以提高这种做法在课堂上的质量。为了实现这一目的，该项目将研究一种新的基于技术的形成性评估工具的有效性，用于课堂辩论-“诊断群体的辩论水平”（DiALoG）-为此，已在实验室环境中进行了心理测量验证工作。DiALoG评估工具允许教师记录课堂谈话并显示多个维度的分数-包括个人内部和人际关系-用于形成性评估目的。该项目将与6 - 8年级的科学教师合作，通过DiALoG仪器生成的实时形成性评估数据来监测和支持论证。DiALoG将与“放大科学”一起使用，这是劳伦斯·霍尔科学开发的课程，结合了从证据中进行论证的科学实践，以及一套新开发的响应式迷你课程（RML），包括20-30分钟的教学策略，旨在帮助教师提供反馈给学生的思考和遵循-直到DiALoG工具确定需要进一步支持的论证事件。本研究将对DiALoG进行改进和扩展，并评估其对教师教学内容知识和形成性评价实践的影响。本研究将解决两个具体的研究问题：（1）如何改进DiALoG，使其成为一个可靠、实用、有用的口语论证形成性评价工具？以及（2）当实施旨在支持口头论证的科学单元时，DiALoG的使用如何影响围绕循证论证的教师形成性评估实践？为了回答这些问题，该项目将对100名教师进行随机对照试验：50名教师将使用DiALoG教授以辩论为重点的课程，50名教师将在没有DiALoG的情况下教授相同的课程。控制和治疗教师都将收到教授三个放大科学课程单元所需的所有数字和物理材料。治疗教师还将获得最新版本的DiALoG，包括相关的RML，以及使用DiALoG与Amplify课程的支持材料。一个重点教师小组（5名来自治疗组，5名来自对照组）将成为额外数据收集和分析的对象。三个重点课程，其中学生正在从事小组或全班口头辩论，将从三个放大科学课程单元中选出。随机对照试验的教师测量将包括经过验证的工具，例如：（a）教师教学内容知识的事前和事后评估;（B）课后和单元后调查，教师将自我报告其形成性评估做法;以及（c）重点教室选定课程的录像。为了观察治疗组和对照组之间形成性评估实践的潜在差异，将使用协议来分析重点教室的视频记录，包括（a）改革后的教学观察协议;（B）课堂内科学论证评估;和（c）教师和学生的形成性评估。这项工作的主要成果将是一个研究知情和现场测试的原型，以提高教学质量和学习论证在中学科学课堂上使用在不同的学习环境。

This is an early-stage design and development collaborative study submitted to the assessment strand of the Discovery Research PreK-12 (DRK-12) program, in response to Program Solicitation NSF 15-592. The fundamental purpose of this project is to support teacher practice and professional learning around oral scientific argumentation in order to improve the quality of this practice in classrooms. To achieve this purpose, the project will examine the validity of a new technology-based formative assessment tool for classroom argumentation--"Diagnosing the Argumentation Levels of Groups" (DiALoG)--for which psychometric validation work has been conducted in a laboratory setting. The DiALoG assessment tool allows teachers to document classroom talk and display scores across multiple dimensions--both intrapersonal and interpersonal--for formative assessment purposes. The project will work with 6th-8th grade science teachers to monitor and support argumentation through real-time formative assessment data generated by the DiALoG instrument. DiALoG will be used in conjunction with "Amplify Science", a Lawrence Hall of Science-developed curriculum that incorporates the science practice of engaging in argument from evidence, and a suite of newly developed Responsive Mini-Lessons (RMLs), which consist of 20-30 minute instructional strategies designed to assist teachers to provide feedback to students' thinking and follow-up to argumentation episodes that the DiALoG tool identifies in need of further support. The study will allow the refinement and expansion of DiALoG and evaluation of its impact on teacher pedagogical content knowledge and formative assessment practices in widespread classroom use.The project will address two specific research questions: (1) How can DiALoG be refined to provide a formative assessment tool for oral argumentation that is reliable, practical, and useful in middle school classrooms?; and (2) How does the use of DiALoG affect teacher formative assessment practices around evidence-based argumentation, when implementing science units designed to support oral argumentation? In order to answer these questions, the project will conduct a randomized control trial with 100 teachers: 50 will teach argumentation-focused curriculum with DiALoG, 50 will teach the same curriculum without DiALoG. Both control and treatment teachers will receive all digital and physical materials needed to teach three Amplify Science curriculum units. Treatment teachers will be provided also with the most recent version of DiALoG, including the linked RMLs, as well as support materials for using DiALoG with the Amplify curriculum. A subgroup of focus teachers (5 from the treatment group, and 5 from the control group) will be the subject of additional data collection and analysis. Three focus lessons, in which students are engaging in small-group or whole-class oral argumentation, will be selected from each of the three Amplify Science curricular units. Teacher measures for the randomized control trial will include validated instruments, such as (a) a pre- and post-assessment of teacher pedagogical content knowledge; (b) post-lesson and post-unit surveys in which teachers will self-report on their formative assessment practices; and (c) video recordings of selected lessons in the focus classrooms. In order to observe potential differences in formative assessment practices between treatment and control, protocols will be used to analyze the video recordings of focus classrooms, including (a) Reformed Teaching Observation Protocol; (b) Assessment of Scientific Argumentation inside the Classroom; and (c) Formative Assessment for Teachers and Students. The key outcome of this work will be a research-informed and field-tested prototype to improve the quality of teaching and learning argumentation in middle school science classrooms usable in different learning environments.