Statistical Methods for Modern Evidence Syntheses with Multiple Biases

点击翻译按钮获取中文摘要

PROJECT SUMMARY/ABSTRACT Meta-analyses critically shape clinical recommendations and policy, but their credibility may be undermined by both within-study biases (e.g., confounding in observational studies) and across-study biases (e.g., filtering by publication bias). These biases can produce meta-analysis estimates that are too large, too small, or in the wrong direction. Scientists, clinicians, and health policymakers are increasingly concerned about these biases given recent empirical evidence that meta-analyses on the same topic can disagree with one another and with the results of systematic replication studies, which are designed to minimize publication bias by having independent investigators repeat published studies. This eroding confidence in the published literature and in meta-analyses represents an epistemic turning point. This proposal develops and empirically validates an innovative, domain-independent statistical framework for quantitatively synthesizing studies subject to within- and across-study biases. Aim 1 will thus develop novel, domain-independent statistical sensitivity analyses will quantify how results of meta-analysis estimates might be shifted by within- and across-study biases, that allow bias-corrected synthesis of studies with these two forms of bias, and that forecast the likely range of results of new studies and the impact of adding them to an existing meta-analysis. The methods will be made broadly accessible via user-friendly websites and R software, and their use will be illustrated in meta-analyses on pediatric obesity. Aim 2 will illustrate the methods' real-world impact and compare their performance to that of existing methods by using the methods to characterize the credibility of Cochrane database meta-analyses. Aim 3 will involve a collaboration with “ManyBabies'', an innovative initiative to conduct conceptual replications of landmark results in developmental psychology. The results of these planned replications will be forecasted using the new methods of Aim 1 as well as existing methods; the forecasts will be compared studies' results after they are conducted, providing real performance benchmarks. Aims 2-3 will also provide online “dashboards” allowing intuitive exploration of the results. The immediate-term goal is to develop methods and software that, unlike existing statistical methods, assess the robustness of a given meta-analysis to the joint effects of within- and across-study biases; that synthesize and compare results of meta-analyses with those of studies subject to less publication bias (e.g., replication studies); and that use potentially biased meta-analyses to plan the optimal design of new studies. The long-term goal is to calibrate confidence in meta-analyses to more swiftly inform scientifically robust conclusions that will improve practice and health policy.