Advancing Methodological Knowledge in STEM Education Research: An Empirical Investigation of Design Parameters for Planning Cluster Randomized Trials in Science Education

认识到需要强有力的工具来加强科学教育研究，该项目正在开发一套资源，以协助研究人员规划科学教育干预措施的分组随机试验。这一全面项目的目标是开发统计资源，以便在科学教育研究中设计和进行严格的分组随机试验(CRT)；提高进行功率分析所需参数范围的准确性；为科学研究界开发可通过免费功率分析软件包获得的统计资源。生物科学课程研究和西密歇根大学的课程开发人员和研究人员合作解决这一缺乏统计资源的问题，专门用于科学教育研究人员和评估者的群组随机试验。认识到关于功率分析估计的大部分工作都是在数学和阅读中进行的，研究小组正在开发一套专门用于科学教育研究的参数估计，以提高CRT的准确性和效率。对科学教育干预研究的荟萃分析和对类内相关性(ICC)和协变量结果相关性(R^2)数据的多水平分析的结果为建立力量分析的估计提供了基础。这项努力开发更准确的ICC、R^2和效应大小(ES)的估计将提高CRT在科学教育研究中的内部效度。通过对进行CRT功率分析所需的所有参数进行经验估算，研究人员和评估者将能够找到在科学教育中规划严格CRT所需参数的估计值。通过免费的功率分析软件包访问这一统计资源，增加了在研究规划中使用对所有三个参数(ICC、R^2、ES)的准确估计的可能性。提高功率方程的精度对大规模科学教育研究的规划和实施具有广泛的影响。

Recognizing the need for powerful tools to enhance studies in science education, the project is developing a set of resources to assist researchers in the planning of cluster-randomized trials of science education interventions. The goals of this full-scale project are to develop the statistical resources to design and conduct rigorous cluster-randomized trials (CRTs) in science education research; increase the accuracy of the range of parameters needed to conduct power analyses; and develop statistical resources for the science research community available through a free power analysis software package. Curriculum developers and researchers from Biological Sciences Curriculum Study and Western Michigan University team-up to address this lack of statistical resources specifically for cluster-randomized trials for science education researchers and evaluators. Recognizing that the majority of the work on power analysis estimates has been conducted in mathematics and reading, the research team is developing of a set of estimates of parameters specifically for science education research that increases the accuracy and improves the efficiency of CRTs. Results from the meta-analysis of research on science education interventions and a multi-level analysis of intra-class correlations (ICC) and covariate outcome correlations (R^2) data provide a foundation for establishing estimates for power analysis. This effort to develop more accurate estimates of ICC, R^2, and effect size (ES) will improve the internal validity of CRTs in science education research. By empirically establishing estimates for the full range of parameters needed to conduct a power analysis for CRTs, researchers and evaluators will be able to find estimates of the parameters necessary to plan rigorous CRTs in science education. Access to this statistical resource through a free power analysis software package, increases the likelihood that accurate estimates of all three parameters (ICC, R^2, ES) will be used in research planning. Improving the accuracy of power equations has broad impact on the planning and conducting of large scale science education research.