Generating Accurate Estimates of Required Sample Size for Multilevel Implementation Studies in Mental Health

项目摘要 该项目填补了一个主要的方法差距，阻止研究人员设计研究与准确的估计， 多层次行为健康实施研究所需样本量的配偶。实施科学 是实现NIMH使命的关键。设计实施研究的一个重要步骤是进行一项 统计功效分析，以确定统计检测感兴趣效应所需的最小样本量。 实施研究的功效分析更为复杂，因为它们需要考虑（a）患者 嵌套在嵌套在组织或其他系统内的提供者内，以及（B）通常 专注于测试（或至少说明）更高级别（例如，组织、临床医生） 较低级别的实施决定因素或战略（例如，患者）的结果。当多层次的权力分析- 分析工具可用于适应这些类型的嵌套研究，这些工具要求研究者具有 三个关键设计参数的先验估计，以确定其研究的适当样本量-组内 相关系数（ICC），效应量，协变量解释的方差比例-这不是rou- 可从已发表的文献中获得，并且不能从小型试点研究中可靠地估计。功率 使用这些设计参数的不准确估计的分析很可能是动力不足的， 并且因此处于不能检测到重要影响的风险，或者功率过大，并且因此浪费LIM， 有限的资源。缺乏这些参数的参考值是该领域的一个基本障碍， 设计参数的微小变化可显著改变有效样本量，从N=300到N=50。 在过去的10年里，NIMH资助了大量的实施研究（N=140）， 我们有机会重新访问这些项目的数据集，以生成多层次的准确估计。 行为健康实施研究的设计参数。我们将使用NIH-RePorter来识别所有NIMH- 资助过去10年进行的行为健康实施研究，并与PI合作， 提取设计参数的目标实施和临床结果，我们将总结和发布， 为外地服务。我们还将生成一个预测模型，使PI能够估计定制的设计参数 他们的新研究的特点。基于我们在Penn NIMH ALACRITY内的初步工作， 中心，本项目将（1）生成设计参数的合并估计值和范围（即，ICC效应量 协变量R2）需要准确估计多水平行为健康实施研究中的样本量， 以及（2）识别预测这些设计参数的大小的研究特征。完成 这项工作将消除一个普遍存在的阻碍实施进展的方法障碍 行为健康科学这项研究将有助于更高质量，更可复制的科学，更有效地 使用NIMH资源，以及更高影响力的实施研究，以提高医疗质量和福祉 每年有数百万人经历精神疾病。

Project Summary This project fills a major methods gap that prevents investigators from designing studies with accurate esti- mates of required sample size for multilevel behavioral health implementation studies. Implementation science is essential to achieving NIMH’s mission. An essential step in designing implementation studies is to conduct a statistical power analysis to determine the minimum sample size required to statistically detect effects of interest. Power analyses for implementation research are more complicated because they need to account for (a) patients nested within providers who are nested within organizations or other systems, and (b) scientific aims that typically focus on testing (or at a minimum accounting for) cross-level effects of higher-level (e.g., organization, clinician) implementation determinants or strategies on lower-level (e.g., patient) outcomes. While multilevel power anal- ysis tools are available to accommodate these types of nested studies, the tools require investigators to have prior estimates of three key design parameters to determine the proper sample size for their study —intraclass correlation coefficient (ICC), effect size, and proportion of variance explained by covariates—which are not rou- tinely available from the published literature and cannot be reliably estimated from small pilot studies. Power analyses that use inaccurate estimates of these design parameters are highly likely to be either underpowered, and consequently at-risk of not detecting important effects, or over-powered, and consequently wasteful of lim- ited resources. Lack of reference values for these parameters is a foundational barrier to the field because even small changes in design parameters can dramatically alter the effective sample size from N=300 to N=50. NIMH has funded a large number of implementation studies during the last 10 years (N=140) which provides an opportunity for us to re-access the datasets from these projects to generate accurate estimates of multilevel design parameters for behavioral health implementation studies. We will use NIH-RePorter to identify all NIMH- funded behavioral health implementation studies conducted during the last 10 years and collaborate with PIs to extract design parameters for targeted implementation and clinical outcomes, which we will summarize and pub- lish for the field. We will also generate a predictive model that enables PIs to estimate design parameters tailored to the characteristics of their new studies. Building on our preliminary work within the Penn NIMH ALACRITY Center, this project will (1) generate pooled estimates and ranges of design parameters (i.e., ICCs, effect sizes, covariate R2) needed to accurately estimate sample size in multilevel behavioral health implementation studies, and (2) identify the study characteristics that predict the magnitude of these design parameters. Completion of this work will remove a ubiquitous methodological barrier that undermines the advancement of implementation science in behavioral health. The study will contribute to higher quality, more replicable science, more efficient use of NIMH resources, and higher impact implementation research to improve healthcare quality and well-being for millions of individuals who experience psychiatric disorders each year.