CAREER: Statistical Power Analysis and Optimal Sample Size Planning for Longitudinal Studies in STEM Education

STEM教育项目通常采用纵向设计来评估多种治疗效果，包括特定时间的效果、平均时间的效果以及效果随时间的变化。在设计纵向研究时，一个重要的考虑因素是功效分析，它概述了确保高概率检测重要效应所需的样本量。然而，没有指导方针或教程来帮助应用研究人员进行这种功率分析。此外，研究人员通常在预算约束下进行纵向研究计划，缺乏在这种约束下计算最佳样本量的方法。该项目的目的是为STEM教育纵向研究的优化设计开发一个全面的统计框架、软件工具、说明性示例和培训材料。具体来说，这项工作开发的统计理论、工具和培训将广泛适用于STEM教育计划的纵向设计，以及健康科学、心理学和公共政策方面的其他社会计划。该项目通过估算STEM教育中常用结果的设计参数，并使用先前STEM教育计划的纵向研究数据说明设计和分析方法，为STEM教育做出了贡献。该项目旨在实现四个综合研究和教育目标。首先，研究者将开发一个统计框架来指导STEM教育纵向实验和准实验研究的功率分析和最佳样本量规划，并将其结果进行比较，以帮助研究者选择最适合其纵向研究的设计和分析方法。接下来，该项目将使用正在进行的和先前的纵向研究的数据，以及STEM教育中常用的结果，对设计参数进行实证估计。该研究将在两个新工具（即PowerUpR-Growth和R Shiny App）中执行公式，并开发相应的软件文档。最后，该项目将开发关于设计和分析纵向STEM教育计划的说明性示例、培训材料和研讨会。统计框架和工具有可能提供一种更实用、更灵活的方式来确定更有效的纵向设计，并协助研究人员评估其STEM教育计划的长期影响。指南、示例、培训材料和研讨会将公开免费提供给STEM教育领域和学科的各种学生、研究人员和从业者群体。这是一个教师早期职业发展计划项目，响应国家科学基金会范围内的活动，提供最负盛名的奖项，以支持有潜力在研究和教育中成为学术榜样的早期职业教师。本项目由美国国家科学基金会EDU核心研究（ECR）项目支持。ECR项目强调在该领域产生基础知识的基础STEM教育研究。投资在至关重要、广泛和持久的关键领域：STEM学习和STEM学习环境，扩大STEM参与，以及STEM劳动力发展。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

STEM education programs often utilize a longitudinal design to evaluate multiple treatment effects of interests, including the effect at a particular time, the average effect over time, and the change of the effect over time. A critical consideration when designing longitudinal studies is a power analysis that outlines the sample sizes needed to ensure a high probability of detecting important effects of interest. However, there are no guidelines or tutorials to help applied researchers conduct such power analyses. In addition, researchers usually plan their longitudinal studies under budget constraints and there is a lack of literature providing methods of calculating optimal sample sizes under such constraints. The purpose of this project is to develop a comprehensive statistical framework, software tools, illustrative examples, and training materials for the optimal design of longitudinal studies in STEM education. Specifically, the statistical theory, tools, and training developed by this work will be broadly applicable to longitudinal designs for STEM education programs, and other social programs in health science, psychology, and public policy. This project contributes to STEM education by estimating design parameters for outcomes commonly used in STEM education and illustrating design and analysis methods using data from prior longitudinal studies of STEM education programs.This project is designed to achieve four integrated research and education goals. First, the investigator will develop a statistical framework to guide the power analysis and optimal sample size planning for longitudinal experimental and quasi-experimental studies in STEM education using all currently available methods, and then compare their results to help researchers select the most appropriate design and analytic methods for their longitudinal studies. Next, the project will develop empirical estimates of the design parameters using data from ongoing and prior longitudinal studies with outcomes commonly used in STEM education. The research will execute the formulas in two new tools (i.e., PowerUpR-Growth and an R Shiny App) and develop accompanying software documentation. Finally, this project will develop illustrative examples, training materials, and workshops on the design and analysis of longitudinal STEM education programs. The statistical framework and tools have the potential to provide a more practical and flexible way to identify more efficient longitudinal designs and assist researchers in evaluating the long-term effects of their STEM education programs. The guidelines, examples, training materials, and workshops will be made publicly and freely accessible to diverse and broad groups of students, researchers, and practitioners across STEM education areas and disciplines. This is a Faculty Early Career Development Program project responsive to a National Science Foundation-wide activity that offers the most prestigious awards in support of early-career faculty who have the potential to serve as academic role models in research and education. This project is supported by NSF's EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.