Empirical Benchmarks of Design Parameters for Group Randomized Trials in Teacher Professional Development Intervention Studies

教师的专业发展是提高学生成绩的行动理论的一个重要组成部分，但这种努力很少得到严格的评估。对专业发展进行强有力的评估的一个问题是，这些方案主要是在学校或地区环境中与教师一起进行的，而不是随机分配到不同的活动中。这就要求评估设计围绕着随机分组试验来进行，而教育评估领域缺乏如何构建这种评估的重要知识。该项目正在开发设计参数的经验基准的估计，以用于构建整个学校的数学专业发展工作的评估。该项目使用的大型数据集，已积累使用的数学知识教学（MKT）措施，以发展在数学教师教学内容知识的变化估计，提供价值，可用于估计样本量和其他设计参数的评估设计。 来自韦恩州立大学教育学院、教育考试服务中心和密歇根州立大学的研究人员正在进行这项研究。该项目利用了从多个专业发展项目中收集的数据，这些项目是在研究人员的主持下使用MKT的数学教师，这些数据是在一个在线教师知识评估系统中收集的，1150多所学校的1000多名教师。来自MKT的数据与数据的共同核心中有关进行专业发展的学校和地区的数据以及MKT研究人员收集的其他数据相结合。研究人员使用随机截距分层线性模型来估计四个不同组随机试验设计的设计参数。他们研究了几组协变量参数对方差分量和相应的组内相关估计的影响。由此产生的参数与学校和地区的条件，其中产生的data.The研究结果在研究和评估设计的专业文献中报告。此外，专业发展讲习班在国家教育研究和评价会议上传播研究结果和研究过程。此外，研究人员还将他们的研究结果纳入了优化设计软件的更新中，该软件被多名研究人员和评估人员用于构建他们自己的研究设计。太频繁，研究人员没有足够的知识，潜在的背景问题周围的学校和地区，他们的工作，使他们应该用来确定他们的研究，如样本量的重要特征的参数的准确估计。这项研究的结果提供了更现实的设计参数，研究人员和评估人员需要进行自己的研究估计。

Professional development of teachers is a strong component in the theory of action of improving student outcomes, yet such efforts are rarely rigorously evaluated. One of the problems of robust evaluation of professional development is that these programs are largely conducted with teachers in their school or district settings rather than being randomly assigned individually to different activities. This requires that the evaluation design be developed around group randomized trials, and the field of educational evaluation lacks important knowledge of how to structure such evaluations. This project is developing estimates of the empirical benchmarks of design parameters to be used to structure evaluations of whole school professional development efforts in mathematics. The project uses the large data sets that have been accumulated using the Mathematical Knowledge of Teaching (MKT) measures to develop estimates of the variation in teacher pedagogical content knowledge in mathematics to provide values that can be used to estimate sample size and other design parameters in evaluation design. Researchers from the College of Education at Wayne State University, the Educational Testing Service and Michigan State University are conducting this research.This project leverages the data collected from multiple professional development projects for mathematics teachers that have used the MKT under the auspices of the researchers that have been collected in an online Teacher Knowledge Assessment System from over 10,000 teachers in over 1150 schools. The data from the MKT is combined with data that is available publically in the Common Core of Data about the schools and districts within which the professional development was conducted and additional data that have been collected by the MKT researchers. The researchers use random intercept hierarchical linear models to estimate the design parameters for four different group random trial designs. They examine the impact of several covariate sets of parameters on the variance components and the corresponding estimates of intraclass correlation. The resulting parameters are reported in association with the conditions of the schools and districts in which the data were generated.The findings from this study are reported in the professional literature on research and evaluation design. In addition, professional development workshops disseminate the findings and the processes used to develop them at national educational research and evaluation meetings. In addition, the researchers include their findings in updates of the Optimal Design software that is used by multiple researchers and evaluators to structure the design of their own studies. Too frequently, researchers do not have sufficient knowledge of the potential contextual issues surrounding the schools and districts in which they work to make accurate estimates of the parameters that they should use to determine important characteristics of their research such as sample size. The findings of this study provide more realistic estimates of the design parameters that researchers and evaluators need to conduct their own studies.