Using Anchoring Vignettes to Improve Measures of Teaching Practice

教师对自己教学实践的看法的自我报告的准确性一直是教育研究和评估中的一个限制。该项目采用一种混合的方法，探讨在多大程度上利用从事六种不同教学实践的教师的小插曲，使教师更准确地自我报告自己的教学实践。兰德公司和其他研究人员与比尔和梅林达·盖茨基金会的有效教学措施推广研究合作，将与4-9年级的220名教师合作。研究问题涉及教师在多大程度上理解小插曲中不同层次所描绘的教学实践，锚定小插曲的使用在多大程度上改善了教师自我报告，教师对小插曲的评分中的信息在多大程度上可以用来改进教师自我评分和训练有素的评分员对这些教师视频的观察，以及小插曲校准的教师自我报告如何与对其他教学结果的估计相关联，如学生成绩或教师评级。研究结果为教学实践条件的研究提供了信息，并有望向教师提供有效的反馈。本项目的研究人员已经确定了重要的教学实践，并开发了经过专家审查的小插曲，描述了这些实践实例化的几个水平的质量。该项目利用了盖茨·梅特项目开展的工作，并从这些教师中收集了课堂教学的调查数据和视频。教师按照小插曲中的描述对教学进行排名，这一排名用于校准他们根据调查对自己的教学进行的自我报告评级。对教师视频的分析被用来提供教师实践的另一种衡量标准。研究人员使用校准后的教学调查评级与教学结果相关联，包括学生成绩的增值模型以及学生和校长评级，以检查校准后的调查在多大程度上可以用作稳健的教学衡量标准。教师评价作为一个政策领域，是教育改革中一个日益重要的主题。与简单的自我报告相比，评价方法能够提供更有效和可靠的教学措施，为当前的教育实践提供了更好的证据。有更多的证据表明，教师经过精选的专业发展，教学实践得到了改善，因此，对这些课程的选择更加知情。这项研究的发现确定了如何以经济高效的方式识别关于教学实践的可靠信息，以补充课堂观察的使用。

The accuracy of self report of teachers' perceptions of their instructional practices has been a limitation in research and evaluation in education. This project is using a mixed methodological approach to address the extent to which the use of vignettes of teachers engaging in six different instructional practices lead to more accurate teacher self-report on their own practices. Researchers from the Rand Corporation and others in partnership with the Bill and Melinda Gates Foundation Measures of Effective Teaching Extension study will work with 220 teachers in grades 4-9. Research questions address the extent to which teachers understand the instructional practices being portrayed at different levels in the vignettes, the extent to which the use of anchoring vignettes improve teacher self-report , the extent to which information in the teachers' ratings of the vignettes can be used to improve the alignment of teachers' self-rating and expertly-trained raters observations of videos of these teachers, and how the vignette-calibrated teacher self-reports correlate with estimates of other outcomes of teaching such as student achievement or ratings of teachers. The research findings inform the research on the conditions of instructional practices and hold promise for effective feedback to teachers.Researchers in this project have identified important instructional practices and have developed expert-vetted vignettes that describe several levels of the quality of the instantiation of these practices. This project leverages the work conducted in the Gates MET project and collects survey data and video of classroom teaching from a sample of those teachers. The teachers rank the teaching as described in the vignettes and this ranking is used to calibrate their self-reported rating of their own teaching from surveys. The analysis of teacher videos is be used to provide another measure of teachers' practices. The researchers use the calibrated survey ratings of teaching to correlate with outcomes of teaching including value-added models of student achievement and student and principal ratings to examine the extent to which the calibrated surveys can be used as robust measures of teaching. Teacher evaluation as a policy area is an increasingly important theme in educational reform. The evaluation methodology that can provide more valid and reliable measures of teaching than simple self-report provides better evidence of the current educational practices. Better evidence of improvements in teaching practices as a result of selected professional development for teachers results in more-informed selection of those offerings. The findings from this study identify ways that robust information about teaching practices can been identified in cost-efficient ways that supplement the use of classroom observations.