Usable Measures of Teacher Understanding: Exploring Diagnostic Models and Topic Analysis as Tools for Assessing Proportional Reasoning for Teaching

与教师和他们的知识相关的一个巨大挑战是，以一种既形成又有意义的方式衡量他们的学习，这与他们对学生的可能影响有关。尽管人们努力定义教师应该掌握的知识，尽管之前为制定良好的衡量标准做出了创新努力，但这一挑战仍然存在。该项目通过专门测量专业发展（PD）项目中发展的知识种类来解决这一挑战，这些知识已被证明对教师的课堂实践和学生的学习很重要。该项目旨在开发一种评估方法，识别教师的学习模式，以帮助推动后续的专业发展。探索研究preK-12项目（DRK-12）旨在通过研究和开发创新资源、模型和工具，显著提高preK-12学生和教师对科学、技术、工程和数学（STEM）的学习和教学。DRK-12计划中的项目建立在STEM教育的基础研究和先前的研究和开发工作的基础上，为拟议的项目提供了理论和实证依据。该项目的总体目标是通过开发一种与各种PD项目所教授的内容和技能相一致的测量方法，寻求一种潜在的变革方法来评估教师比例知识。这个工具将基于一种建立在新兴心理测量模型上的新方法。具体来说，诊断分类模型（dcm）将用于诊断教师在PD项目中的学习情况，并用于识别教师学习的进展。统计主题模型（STMs）将用于寻找从开放式回答中出现的理解模式，并为教师的推理提供自然语言洞察力。根据预测效度阶段的结果，将为全国样本构建评估的最终版本，该版本将在参加各种针对比例推理的PD项目的教师中进行测试。这个项目对评价的建立和教师教育具有广泛的影响。它将提供关于教师是否有明确的学习进度的见解。虽然对学生的学习进展已经做了很多工作，但对教师如何学习知之甚少。另一个含义是，STM方法允许机器以一种突出推理中的优点和缺点的方式对自然语言进行评分，而不是简单地返回一个分数。对于形成性的使用，这些信息更有帮助，因为它突出了需要进一步指导的领域。第三个含义是，dcm将允许以比通常可用的更细粒度的理解来评估教师的知识，从而允许对PD进行仔细的细化，以及显示PD整体增长的工具。第四个含义是，将采用一种更系统的方法来获取教师所需的各种知识。使用dcm和STMs开发的评估有可能作为在其他STEM内容领域开发进一步工具的模型。这样的评估不仅有潜力帮助确定成功的PD项目，而且还为PD提供者提供丰富的数据，他们可以从中做出教学决策。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

One of the great challenges related to teachers and their knowledge is measuring their learning in ways that are both formative and meaningful in relation to their likely impact on students. This challenge persists despite efforts to define the knowledge teachers should have and despite previous innovative efforts to create good measures. This project tackles the challenge by specifically aiming to measure the kinds of knowledge developed in professional development (PD) programs that has been shown to matter for teachers' classroom practices and their students' learning. The project aims to develop an assessment that identifies patterns in the teachers' learning in a way that helps drive subsequent professional development. The Discovery Research preK-12 program (DRK-12) seeks to significantly enhance the learning and teaching of science, technology, engineering and mathematics (STEM) by preK-12 students and teachers, through research and development of innovative resources, models and tools. Projects in the DRK-12 program build on fundamental research in STEM education and prior research and development efforts that provide theoretical and empirical justification for proposed projects. The overall goal of this project is to pursue a potentially transformative approach to the assessment of teacher proportional knowledge by developing a measure that is well aligned with the content and skills taught in various PD programs. This instrument will be based on a new approach that builds on emerging psychometric models. Specifically, diagnostic classification models (DCMs) will be utilized to diagnose teachers' learning during a PD program as well as employed to identify the progression in teachers' learning. Statistical topic models (STMs) will be used to look for patterns of understanding that emerge from open-ended responses and provide natural-language insight into teachers' reasoning. A final version of the assessment will be constructed for a national sample based on the results from the predictive validity stage, and this version will be tested with teachers who participate in various types of PD programs targeting proportional reasoning. This project has broad implications for the creation of assessments and for teacher education. It will provide insights about whether there is a clear learning progression for teachers. While much work has been done with students' learning progression, much less is known about how teachers learn. Another implication is that the STM approach allows machine scoring of natural language in a way that highlights strengths and weaknesses in reasoning rather than simply returning a score. For formative use, this is information that is more helpful as it highlights areas for further instruction. A third implication is that DCMs will allow to assess teacher knowledge at a finer-grained understanding than is typically available, thus allowing for careful refinement of PD as well as a tool for showing overall growth in PD. A fourth implication is that a more systematic approach will be followed to capture the kinds of knowledge teachers need. Assessments developed using DCMs and STMs have the potential to serve as models for developing further instruments in other STEM content areas. Such assessments have the potential to not only help identify successful PD programs, but also to provide PD providers with rich data from which they can make instructional decisions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.