Development and evaluation of R-squared effect size measures and methods for mixed effects models

点击翻译按钮获取中文摘要

Hierarchical data structures are ubiquitous in many areas of scientific research, and are particularly common in psychology. Typical examples include students nested within schools, or repeated observations nested within individuals. Such data are often analyzed with mixed effects models, which extend standard regression models by allowing coefficients to vary by cluster (e.g., school). Despite their increasing popularity in psychology and other disciplines, one major limitation in the use of mixed effects models is that researchers rarely report measures of effect size (a quantitative indication of the magnitude of effects/ relationships among variables) and instead typically summarize results primarily with p-values. This stands in sharp contrast to widespread recommendations to convey practical significance rather than relying exclusively on statistical significance. In particular, one effect size measure routinely reported in standard regression-R-squared (R2), reflecting the proportion of variance explained-is rarely reported for mixed effects models. Computing R2 in mixed effects models is markedly more complex than in standard regression, and there has been confusion about how best to do so. In my previous work, I explained how the methodological literature on mixed effect model R2 has historically suffered from many shortcomings, leading to researchers either not reporting such measures at all or, perhaps more troublingly, using and interpreting them incorrectly. To address these shortcomings, I provided a unifying framework to defining, computing, and interpreting R2 for mixed effects models (Rights & Sterba, 2019, in press). Several key shortcomings persist, however, in that (a) this framework does not apply to every possible model specification, (b) little work has been done to assess the statistical properties of the available mixed effect R2 measures, and (c) there is a lack of available software to compute measures. In this proposal, I aim to address these shortcomings. Objective 1 is to extend the Rights & Sterba R2 framework to a wider variety of modeling contexts, such as generalized linear mixed effects models, cross-classified models, longitudinal models with complex residual variance structures, and latent variable models. Objective 2 is to formally evaluate both preexisting and newly developed R2 methods in terms of bias, efficiency, and coverage via simulation. Different methods of bias adjustment and confidence interval construction will be considered and developed. Objective 3 is to ease the computational burden on researchers by providing open-source software to compute these measures and implement these methods. Overall, the long-term aim of this research program is to enable researchers to better compute effect size for mixed effects models. This responds to widespread recommendations to consider practical significance of statistical results, and will be useful in many research disciplines in psychology and beyond.