CAREER: Design of Multisite Moderation Studies to Examine the Factors that Explain Treatment Effect Variation on Student Mathematics Achievement in Teacher Professional Development

这是一个教师早期职业发展（CAREER）项目提交给EHR核心研究（ECR）计划，响应计划征求NSF 15-555。它的目的是开发一个全面的框架，通知和调查的治疗效果的变化，在教师专业发展（PD）的研究，使用多站点适度研究的变量，影响治疗变量和结果变量之间的关系的方向和幅度。该研究将实现四个综合研究和教育目标。第一个目标将开发的统计理论的基础上的有效和高效的设计多站点温和的专业发展的研究，并推导出公式的统计功率和最小可检测的效果大小差异与置信区间的主持人的影响，在两个和三个级别的多站点随机和非随机研究。第二个目标将应用一个框架和统计理论，以经验估计现有的小学数学专家（EMS）计划的主持人的影响和设计参数，并演示了计算的统计能力和最小可检测的效果大小差异，这个EMS计划使用的公式来自第一个目标。第三个目标是在现有的免费软件中执行从第一个doal导出的公式，并开发相应的软件文档。最后，第四个目标将提供研究生和研究人员PD使用本项目的结果来设计多站点适度研究。它将包括开发课程和教学材料的工具，并使用这些材料进行研讨会。拟议的研究建立在一个正在进行的NSF资助的数学教师PD研究的数据上，该研究考察了EMS认证的教师与EMS非认证的教师在学校或地区内匹配的平均效果。CAREER研究将从正在进行的关于学生数学成绩的工作中收集二级数据;教师的数学内容知识;协变量和潜在的调节因素，如学区类型，学校，教师和学生背景信息。数据分析将集中在学生的数学成绩收益的调节效应的经验估计，使用两个和三个层次的分层线性模型进行适度分析;经验估计的设计参数的功率分析;和计算的功率和最小可检测的效果大小差异的EMS程序的演示。同样，该项目还将开发软件和文件，使应用研究人员能够广泛使用复杂的统计概念，并使其切实可行，以开展本研究所开展的工作;该项目还将开发培训材料和设计专业发展讲习班。该项目将通过咨询委员会和软件及PD材料的用户进行内部和外部评估。这项工作的主要成果将是一个框架和工具，以提高研究人员的能力，以确定解释教师专业发展中的治疗效果差异的因素和条件;因此，在回答“为谁工作，在什么情况下工作”的问题时，通过研究结果的严格证据来提高教师的效力，并解释和探索治疗效果变化的来源。

This is a Faculty Early Career Development (CAREER) project submitted to the EHR Core Research (ECR) program, responsive to Program Solicitation NSF 15-555. Its purpose is to develop a comprehensive framework to inform and investigate treatment effect variation in teacher professional development (PD) studies using multisite moderation studies focused on the variables that affect the direction and magnitude of the relation between the treatment variable and the outcome variable. The study will accomplish four integrated research and education goals. The first goal will develop the statistical theory underlying the effective and efficient design of multisite moderation studies of professional development, and derive formulas for the statistical power and the minimum detectable effect size difference with confidence intervals for moderator effects in two- and three-level multisite randomized and nonrandomized studies. The second goal will apply a framework and statistical theory to empirically estimate the moderator effects and design parameters of existing Elementary Mathematics Specialists (EMS) programs, and demonstrate the calculation of the statistical power and the minimum detectable effect size difference for this EMS program using the formulas derived from the first goal. The third goal will execute the formulas derived from the first doal in the existing free-available software, and develop accompanying software documentation. Finally, the fourth goal will provide graduate students and researchers PD in using the results from this project to design multisite moderation studies. It will include developing curricular and instructional materials on the tools and conducting workshops using these materials.The proposed research builds on data from an ongoing NSF-funded mathematics teacher PD study, which examines the average effects of the EMS certified teachers with the EMS non-certified teachers that are matched within schools or districts. The CAREER study will gather secondary data from the ongoing work on students' mathematics achievement; teachers' mathematical content knowledge; and covariates and potential moderators, such as type of school district, school, teacher, and student background information. Data analyses will focus on students' mathematics achievement gains empirical estimates of moderator effects using two- and three-level hierarchical linear modeling for moderation analyses; empirical estimates of design parameters necessary for power analyses; and demonstration of calculating the power and the minimum detectable effect size difference for this EMS program. Likewise, the project will develop software and documentation that make the complex statistical concepts broadly accessible and practical to applied researchers to implement the work developed from this study; as well as the training materials and design of the professional development workshops. The project will be evaluated both internally and externally through an advisory board and users of the software and PD materials. The main outcome of this effort will be a framework and tools to increase researchers' capacity to identify the factors and conditions that explain the treatment effect variation in teacher professional development; thus, improving teacher effectiveness through the rigorous evidence of research findings in response to questions regarding "work for whom, and under what circumstances", and interpreting and exploring the sources of the treatment effect variations.