Development and evaluation of R-squared effect size measures and methods for mixed effects models

混合效应模型的 R 平方效应大小测量和方法的开发和评估

• 批准号: RGPIN-2020-06132
• 负责人: Rights, Jason
• 金额: $ 1.31万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=764124
• 关键词: Development; evaluation; squared; effect; size

Hierarchical data structures are ubiquitous in many areas of scientific research, and are particularly common in psychology. Typical examples include students nested within schools, or repeated observations nested within individuals. Such data are often analyzed with mixed effects models, which extend standard regression models by allowing coefficients to vary by cluster (e.g., school). Despite their increasing popularity in psychology and other disciplines, one major limitation in the use of mixed effects models is that researchers rarely report measures of effect size (a quantitative indication of the magnitude of effects/ relationships among variables) and instead typically summarize results primarily with p-values. This stands in sharp contrast to widespread recommendations to convey practical significance rather than relying exclusively on statistical significance. In particular, one effect size measure routinely reported in standard regression-R-squared (R2), reflecting the proportion of variance explained-is rarely reported for mixed effects models. Computing R2 in mixed effects models is markedly more complex than in standard regression, and there has been confusion about how best to do so. In my previous work, I explained how the methodological literature on mixed effect model R2 has historically suffered from many shortcomings, leading to researchers either not reporting such measures at all or, perhaps more troublingly, using and interpreting them incorrectly. To address these shortcomings, I provided a unifying framework to defining, computing, and interpreting R2 for mixed effects models (Rights & Sterba, 2019, in press). Several key shortcomings persist, however, in that (a) this framework does not apply to every possible model specification, (b) little work has been done to assess the statistical properties of the available mixed effect R2 measures, and (c) there is a lack of available software to compute measures. In this proposal, I aim to address these shortcomings. Objective 1 is to extend the Rights & Sterba R2 framework to a wider variety of modeling contexts, such as generalized linear mixed effects models, cross-classified models, longitudinal models with complex residual variance structures, and latent variable models. Objective 2 is to formally evaluate both preexisting and newly developed R2 methods in terms of bias, efficiency, and coverage via simulation. Different methods of bias adjustment and confidence interval construction will be considered and developed. Objective 3 is to ease the computational burden on researchers by providing open-source software to compute these measures and implement these methods. Overall, the long-term aim of this research program is to enable researchers to better compute effect size for mixed effects models. This responds to widespread recommendations to consider practical significance of statistical results, and will be useful in many research disciplines in psychology and beyond.

分层数据结构在科学研究的许多领域中无处不在，在心理学中尤其常见。典型的例子包括学生嵌套在学校，或重复观察嵌套在个人。这些数据通常用混合效应模型进行分析，混合效应模型通过允许系数随聚类而变化来扩展标准回归模型（例如，学校）。尽管混合效应模型在心理学和其他学科中越来越受欢迎，但使用混合效应模型的一个主要限制是研究人员很少报告效应大小（变量之间效应/关系大小的定量指标），而是通常主要用p值总结结果。这与广泛建议传达实际意义而不是完全依赖统计意义形成鲜明对比。特别是，在标准回归中常规报告的一个效应量测量-R平方（R2），反映了解释的方差比例-很少报告混合效应模型。在混合效应模型中计算R2明显比标准回归更复杂，并且对于如何最好地做到这一点一直存在困惑。在我以前的工作中，我解释了混合效应模型R2的方法论文献在历史上是如何存在许多缺点的，导致研究人员要么根本没有报告这些措施，要么更麻烦的是，错误地使用和解释它们。为了解决这些缺点，我提供了一个统一的框架来定义，计算和解释混合效应模型的R2（Rights & Sterba，2019，出版中）。然而，几个关键的缺点仍然存在，因为（a）这个框架并不适用于每一个可能的模型规格，（B）几乎没有做什么工作来评估可用的混合效应R2测量的统计特性，以及（c）缺乏可用的软件来计算测量。在本建议中，我旨在解决这些缺点。目标1是将Rights & Sterba R2框架扩展到更广泛的建模环境，例如广义线性混合效应模型，交叉分类模型，具有复杂残差方差结构的纵向模型和潜变量模型。目标2是通过模拟正式评估先前存在的和新开发的R2方法的偏倚、效率和覆盖率。将考虑并开发不同的偏差调整和置信区间构建方法。目标3是通过提供开源软件来计算这些措施并实施这些方法，从而减轻研究人员的计算负担。总的来说，这项研究计划的长期目标是使研究人员能够更好地计算混合效应模型的效应大小。这响应了广泛的建议，考虑统计结果的实际意义，并将在心理学和超越许多研究学科有用。