CAREER: Design of Multisite Moderation Studies to Examine the Factors that Explain Treatment Effect Variation on Student Mathematics Achievement in Teacher Professional Development

这是一个提交给EHR核心研究（ECR）项目的教师早期职业发展（Career）项目，响应项目征集NSF 15-555。其目的是建立一个全面的框架，以了解和调查教师专业发展（PD）研究中的治疗效果变化，使用多站点调节研究，重点关注影响治疗变量和结果变量之间关系的方向和大小的变量。该研究将完成四个综合研究和教育目标。第一个目标将发展有效和高效设计多地点专业发展调节研究的统计理论，并推导出两级和三级多地点随机和非随机研究中调节效应的统计能力和最小可检测效应大小差异的置信区间的公式。第二个目标将应用一个框架和统计理论来经验性地估计现有小学数学专家（EMS）计划的调节效应和设计参数，并使用从第一个目标导出的公式来证明该EMS计划的统计功率和最小可检测效应大小差异的计算。第三个目标将在现有的免费软件中执行从第一个目标衍生出来的公式，并开发相应的软件文档。最后，第四个目标将为研究生和研究人员提供使用该项目结果设计多站点调节研究的PD。它将包括编写有关这些工具的课程和教学材料，并利用这些材料举办讲习班。拟议的研究建立在一项正在进行的美国国家科学基金会资助的数学教师PD研究的数据基础上，该研究检查了EMS认证教师与EMS非认证教师在学校或地区内匹配的平均效果。CAREER研究将从正在进行的关于学生数学成绩的研究中收集二手数据；教师的数学内容知识；协变量和潜在的调节因子，如学区类型、学校、教师和学生背景信息。数据分析将关注学生的数学成绩收益，使用二级和三级层次线性模型进行调节分析，对调节效应进行实证估计；功率分析所需的设计参数的经验估计；并演示了计算该EMS程序的功率和最小可检测效应大小差异。同样，该项目将开发软件和文件，使复杂的统计概念对应用研究人员广泛可及和实用，以执行从本研究中发展出来的工作；以及培训材料和专业发展工作坊的设计。该项目将通过咨询委员会和软件和PD材料的用户进行内部和外部评估。这项工作的主要成果将是一个框架和工具，以提高研究人员识别解释教师专业发展中治疗效果变化的因素和条件的能力；因此，在回答“为谁工作，在什么情况下工作”的问题时，通过研究结果的严格证据来提高教师的有效性，并解释和探索治疗效果变化的来源。

This is a Faculty Early Career Development (CAREER) project submitted to the EHR Core Research (ECR) program, responsive to Program Solicitation NSF 15-555. Its purpose is to develop a comprehensive framework to inform and investigate treatment effect variation in teacher professional development (PD) studies using multisite moderation studies focused on the variables that affect the direction and magnitude of the relation between the treatment variable and the outcome variable. The study will accomplish four integrated research and education goals. The first goal will develop the statistical theory underlying the effective and efficient design of multisite moderation studies of professional development, and derive formulas for the statistical power and the minimum detectable effect size difference with confidence intervals for moderator effects in two- and three-level multisite randomized and nonrandomized studies. The second goal will apply a framework and statistical theory to empirically estimate the moderator effects and design parameters of existing Elementary Mathematics Specialists (EMS) programs, and demonstrate the calculation of the statistical power and the minimum detectable effect size difference for this EMS program using the formulas derived from the first goal. The third goal will execute the formulas derived from the first doal in the existing free-available software, and develop accompanying software documentation. Finally, the fourth goal will provide graduate students and researchers PD in using the results from this project to design multisite moderation studies. It will include developing curricular and instructional materials on the tools and conducting workshops using these materials.The proposed research builds on data from an ongoing NSF-funded mathematics teacher PD study, which examines the average effects of the EMS certified teachers with the EMS non-certified teachers that are matched within schools or districts. The CAREER study will gather secondary data from the ongoing work on students' mathematics achievement; teachers' mathematical content knowledge; and covariates and potential moderators, such as type of school district, school, teacher, and student background information. Data analyses will focus on students' mathematics achievement gains empirical estimates of moderator effects using two- and three-level hierarchical linear modeling for moderation analyses; empirical estimates of design parameters necessary for power analyses; and demonstration of calculating the power and the minimum detectable effect size difference for this EMS program. Likewise, the project will develop software and documentation that make the complex statistical concepts broadly accessible and practical to applied researchers to implement the work developed from this study; as well as the training materials and design of the professional development workshops. The project will be evaluated both internally and externally through an advisory board and users of the software and PD materials. The main outcome of this effort will be a framework and tools to increase researchers' capacity to identify the factors and conditions that explain the treatment effect variation in teacher professional development; thus, improving teacher effectiveness through the rigorous evidence of research findings in response to questions regarding "work for whom, and under what circumstances", and interpreting and exploring the sources of the treatment effect variations.