A Statistical Framework and Tools for Planning Multilevel Randomized Cost-Effectiveness Trials

该研究项目由EHR核心研究（ECR）计划提供支持，该计划支持推进STEM学习和学习环境的基础研究，扩大STEM参与和STEM劳动力发展。对STEM教育计划，政策和实践的影响进行更全面评估的需求不断增加。一般来说，成本效益研究的目的是通过评估替代方案的比较性能，在其有效性和产生这些效果的成本，以满足这一需求。目前的评估几乎只涉及计划的影响。需要新的和更全面的评估工具，不仅要详细说明STEM项目的影响，还要详细说明在许多组织层面产生这些影响的净成本，这些影响需要弥补学校系统（教室，学校和地区）。这项研究开发了一个统计框架和软件，大大增强了STEM领域多层次成本效益研究的设计（估计和分离这些层次的成本）。本研究的目的有四个方面：（1）开发一个统计框架，指导设计的成本效益研究的权力，样本量和最小可检测效应量（MDES）;（2）评估和证实的精度和效用的结果公式通过蒙特卡罗模拟;（3）实施成本计算公式在一个访问和用户友好的软件程序，PowerUp！。实际上，生成一个新的更新版本的软件，包括多层次的成本效益的能力;（4）开发一个参数值的经验目录，以指导研究人员在明智地实施这些分析。该项目的范围包括广泛的多层次设计，包括十个最常见的多层次研究设计，并将其扩展到合理的成本效益分析。从概念上讲，这些设计沿着三个主要维度变化：（1）集群级别（教室或学校）的数量;（2）计划分配的级别;以及（3）可用的成本信息的级别。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。

This research project is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development.There continues to be increasing demand for more comprehensive assessments of the impacts of STEM education programs, policies, and practices. In general, cost-effectiveness studies are designed to address this demand by evaluating the comparative performance of alternative programs in terms of both their effectiveness and the cost of producing those effects. Current assessments have dealt almost exclusively with program effects. New and more comprehensive assessment tools are needed to detail not just the effects of STEM programs but also the net cost of producing those impacts across the many organizational levels take make up school systems (classrooms, schools, and districts). This research develops a statistical framework and software that critically enhance the design of multilevel cost-effectiveness studies in STEM areas (estimating and separating costs across these levels). The purpose of this study is fourfold: (1) to develop a statistical framework that guides the design of cost-effectiveness studies in terms of power, sample sizes, and minimum detectable effect size (MDES); (2) to assess and substantiate the precision and utility of the resulting formulas through Monte Carlo simulations; (3) to implement the costing formulas in an accessible and user-friendly software program, PowerUp!. In effect generating a new updated version of the software that includes the capacity for multilevel cost effectiveness; and (4) to develop an empirical catalogue of parameter values to guide researchers in the judicious implementation of these analyses. The scope of this project includes a broad range of multilevel designs, including ten of the most common multilevel research designs, and extends them to sensibly include cost effectiveness analyses. Conceptually, these designs vary along three primary dimensions: (1) the number of cluster levels (classrooms, or schools); (2) the level of program assignment; and (3) the level of cost information available. The statistical framework and tools for the effective and efficient design of multisite randomized trials will explicitly address each of these dimensions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.