A Statistical Framework and Tools for Planning Multilevel Randomized Cost-Effectiveness Trials

这项研究项目得到了EHR核心研究(ECR)计划的支持，该计划支持推进STEM学习和学习环境的基础研究、扩大对STEM的参与以及STEM劳动力发展的工作。对STEM教育计划、政策和实践的影响进行更全面评估的需求不断增加。一般来说，成本效益研究旨在通过评估替代方案在有效性和产生这些效果的成本方面的比较表现来满足这一需求。目前的评估几乎完全涉及项目效果。需要新的、更全面的评估工具，不仅要详细说明STEM计划的影响，还要详细说明在学校系统(教室、学校和学区)的许多组织级别上产生这些影响的净成本。这项研究开发了一个统计框架和软件，极大地加强了STEM地区多水平成本效益研究的设计(估算和划分这些水平的成本)。这项研究的目的有四个方面：(1)开发一个统计框架，从功率、样本大小和最小可检测效应大小(MDES)方面指导成本效益研究的设计；(2)通过蒙特卡罗模拟评估和证实所得公式的精确度和实用性；(3)在可访问和用户友好的软件程序PowerUp！中实施成本计算公式。实际上是生成一个新的软件更新版，其中包括多级成本效益的能力；以及(4)编制一个参数值的经验目录，以指导研究人员明智地执行这些分析。该项目的范围包括广泛的多水平设计，包括十个最常见的多水平研究设计，并将其合理地扩展到包括成本效益分析。从概念上讲，这些设计沿着三个主要维度变化：(1)群组级别(教室或学校)的数量；(2)计划分配的级别；以及(3)可用成本信息的级别。有效和高效的多点随机试验设计的统计框架和工具将明确处理这些维度中的每一个。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This research project is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development.There continues to be increasing demand for more comprehensive assessments of the impacts of STEM education programs, policies, and practices. In general, cost-effectiveness studies are designed to address this demand by evaluating the comparative performance of alternative programs in terms of both their effectiveness and the cost of producing those effects. Current assessments have dealt almost exclusively with program effects. New and more comprehensive assessment tools are needed to detail not just the effects of STEM programs but also the net cost of producing those impacts across the many organizational levels take make up school systems (classrooms, schools, and districts). This research develops a statistical framework and software that critically enhance the design of multilevel cost-effectiveness studies in STEM areas (estimating and separating costs across these levels). The purpose of this study is fourfold: (1) to develop a statistical framework that guides the design of cost-effectiveness studies in terms of power, sample sizes, and minimum detectable effect size (MDES); (2) to assess and substantiate the precision and utility of the resulting formulas through Monte Carlo simulations; (3) to implement the costing formulas in an accessible and user-friendly software program, PowerUp!. In effect generating a new updated version of the software that includes the capacity for multilevel cost effectiveness; and (4) to develop an empirical catalogue of parameter values to guide researchers in the judicious implementation of these analyses. The scope of this project includes a broad range of multilevel designs, including ten of the most common multilevel research designs, and extends them to sensibly include cost effectiveness analyses. Conceptually, these designs vary along three primary dimensions: (1) the number of cluster levels (classrooms, or schools); (2) the level of program assignment; and (3) the level of cost information available. The statistical framework and tools for the effective and efficient design of multisite randomized trials will explicitly address each of these dimensions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.