Development and evaluation of R-squared effect size measures and methods for mixed effects models

层次化数据结构在许多科学研究领域中普遍存在，在心理学中尤其常见。典型的例子包括学生嵌套在学校里，或者重复观察嵌套在个人身上。这类数据通常使用混合效应模型进行分析，该模型通过允许系数随集群(例如，学校)而变化来扩展标准回归模型。尽管混合效应模型在心理学和其他学科中越来越受欢迎，但使用混合效应模型的一个主要限制是，研究人员很少报告效应大小的度量(效应大小/变量之间关系的量化指示)，而通常主要使用p值来总结结果。这与广泛提出的传达实际意义而不是完全依赖统计意义的建议形成鲜明对比。特别是，在混合效应模型中，通常以标准回归R平方(R2)报告的一种效应大小度量，反映了所解释的方差的比例，很少被报道。在混合效应模型中计算R2明显比在标准回归中计算R2复杂得多，人们一直对如何最好地计算R2感到困惑。 在我之前的工作中，我解释了关于混合效应模型R2的方法论文献历来存在许多缺陷，导致研究人员要么根本没有报告这些衡量标准，要么可能更麻烦地使用和解释它们不正确。为了解决这些缺点，我提供了一个统一的框架来定义、计算和解释混合效果模型的R2(Rights&Sterba，2019年出版)。然而，仍然存在几个主要缺点，原因是：(A)这一框架并不适用于所有可能的模型规格；(B)在评估现有的混合效应R2措施的统计性质方面所做的工作很少；以及(C)缺乏计算措施的现有软件。 在这项提案中，我的目标是解决这些缺点。目标1是将Rights&Sterba R2框架扩展到更广泛的建模环境，如广义线性混合效应模型、交叉分类模型、具有复杂残差方差结构的纵向模型和潜变量模型。目标2是通过模拟对现有的和新开发的R2方法在偏差、效率和覆盖率方面进行正式评估。将考虑和开发不同的偏差调整和可信区间构建方法。目标3是通过提供开源软件来计算这些测量并实现这些方法，从而减轻研究人员的计算负担。总体而言，这项研究计划的长期目标是使研究人员能够更好地计算混合效应模型的效应大小。这回应了考虑统计结果的实际意义的广泛建议，并将在心理学和其他领域的许多研究学科中有用。

Hierarchical data structures are ubiquitous in many areas of scientific research, and are particularly common in psychology. Typical examples include students nested within schools, or repeated observations nested within individuals. Such data are often analyzed with mixed effects models, which extend standard regression models by allowing coefficients to vary by cluster (e.g., school). Despite their increasing popularity in psychology and other disciplines, one major limitation in the use of mixed effects models is that researchers rarely report measures of effect size (a quantitative indication of the magnitude of effects/ relationships among variables) and instead typically summarize results primarily with p-values. This stands in sharp contrast to widespread recommendations to convey practical significance rather than relying exclusively on statistical significance. In particular, one effect size measure routinely reported in standard regressionR-squared (R2), reflecting the proportion of variance explainedis rarely reported for mixed effects models. Computing R2 in mixed effects models is markedly more complex than in standard regression, and there has been confusion about how best to do so. In my previous work, I explained how the methodological literature on mixed effect model R2 has historically suffered from many shortcomings, leading to researchers either not reporting such measures at all or, perhaps more troublingly, using and interpreting them incorrectly. To address these shortcomings, I provided a unifying framework to defining, computing, and interpreting R2 for mixed effects models (Rights & Sterba, 2019, in press). Several key shortcomings persist, however, in that (a) this framework does not apply to every possible model specification, (b) little work has been done to assess the statistical properties of the available mixed effect R2 measures, and (c) there is a lack of available software to compute measures. In this proposal, I aim to address these shortcomings. Objective 1 is to extend the Rights & Sterba R2 framework to a wider variety of modeling contexts, such as generalized linear mixed effects models, cross-classified models, longitudinal models with complex residual variance structures, and latent variable models. Objective 2 is to formally evaluate both preexisting and newly developed R2 methods in terms of bias, efficiency, and coverage via simulation. Different methods of bias adjustment and confidence interval construction will be considered and developed. Objective 3 is to ease the computational burden on researchers by providing open-source software to compute these measures and implement these methods. Overall, the long-term aim of this research program is to enable researchers to better compute effect size for mixed effects models. This responds to widespread recommendations to consider practical significance of statistical results, and will be useful in many research disciplines in psychology and beyond.