Improving Reproducibility by Incorporating Uncertainty

通过纳入不确定性来提高再现性

• 批准号: 10322751
• 负责人: Timothy L. Lash
• 金额: $ 27.05万
• 依托单位: EMORY UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-02-06 至 2024-01-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10322751
• 关键词: Address; Appearance; Area; Bayesian Method; Behavior; Body mass index; CYP2D6 gene; Clinical; Communities; Consensus; Creativeness; Data Collection; Effectiveness; Evaluation; Genotype; Grant; Guidelines; Heterogeneity; Individual; Influentials; Informatics; Journals; Judgment; Life Style; Measurement; Meta-Analysis; Methods; Obesity; Outcome; Paper; Perception; Predisposition; Publication Bias; Reaction; Relative Risks; Reporting; Reproducibility; Research; Research Personnel; Residual state; Resources; Risk; Selection Bias; Source; Speed; Systematic Bias; Tamoxifen; Techniques; Testing; Uncertainty; United States National Institutes of Health; advanced analytics; base; cancer recurrence; complex data; disorder later incidence prevention; editorial; evidence base; improved; informatics tool; malignant breast neoplasm; metabolic phenotype; mortality; simulation; skills; web-enabled

In recent years, stakeholders in the scientific and lay communities have raised alarms about a lack of reproducibility of scientific results. These stakeholders view the reproducibility crisis as a product of the behavior of researchers and editors. While these behaviors likely have an impact on reproducibility, there are credible reasons why studies of complex data should be expected to arrive at different estimates. First, each study is differentially susceptible to systematic biases, including confounding, selection bias and measurement error. These biases may be large drivers of the appearance of poor reproducibility. Second, many studies that have been criticized for lack of replication are small, and therefore subject to substantial random variability. In combination with selection forces emanating from significance testing, these small studies are likely to overestimate effects, further contributing to the appearance of poor reproducibility. To date, proposed solutions to the perceived reproducibility crisis have largely ignored these contributing factors. We propose to (a) use simulation-based quantitative bias analysis techniques to adjust for the influence of systematic errors on estimates of association and on summaries of an evidence base, and (b) use Bayesian statistical methods to synthesize prior information with estimates of association and summaries of an evidence base to reduce random variability. The premise of the proposed project is that the use of these informatics approaches will reduce the potential for systematic and random error to misleadingly portray research as poorly reproducible and will identify the most important limitations in an evidence base, which will optimize decisions regarding new data collection. The proposed informatics will be extended and applied in the context of two high profile, controversial topic areas with complex data, which will provide examples applicable to other topic areas. For both topic areas, many sources of potential bias have been identified in the surrounding discourse—as have powerful sources of prior information to temper uncertainty—but their influences on individual estimates of association and summaries of the evidence base have not been fully quantified. Quantitative adjustments for these errors using quantitative bias analysis and Bayesian methods—and for publication bias on meta-analytic summaries—would improve reproducibility. We will then extend and apply web-enabled informatics tools to implement the methods for any topic with a set of heterogeneous study results, allowing stakeholders without advanced analytic skills to tailor the underlying assumptions and see for themselves the impact on the summary results. By achieving our aims, this project will advance the use of research informatics to diminish the reproducibility crisis, help to speed consensus-building for any research topic, and productively channel research resources towards resolving the most influential sources of uncertainty in any topic area.

近年来，科学界和非专业界的利益相关者对缺乏科学知识提出了警告。 科学成果的可重复性。这些利益相关者将再现性危机视为 研究人员和编辑的行为。虽然这些行为可能会影响可重复性，但 为什么对复杂数据的研究会得出不同的估计数。一是各 研究对系统偏倚的敏感性不同，包括混杂、选择偏倚和测量 错误.这些偏差可能是再现性差的主要驱动因素。第二，许多研究表明， 已经被批评为缺乏复制是小的，因此受到很大的随机变化。在 结合显著性检验产生的选择力，这些小型研究可能会 过高估计效果，进一步导致再现性差的外观。迄今为止， 在很大程度上忽略了这些影响因素。我们建议（a）使用 基于模拟的定量偏差分析技术，以调整系统误差对 关联估计和证据基础的总结，以及（B）使用贝叶斯统计方法， 将先验信息与关联估计和证据库摘要进行综合， 随机变异性拟议项目的前提是，使用这些信息学方法将 减少系统性和随机性错误的可能性，从而误导性地将研究描述为可重复性差 并将确定证据库中最重要的限制，这将优化有关新的 数据收集。拟议的信息学将在两个高姿态的背景下得到扩展和应用， 具有复杂数据的有争议的主题领域，将提供适用于其他主题领域的示例。为 这两个主题领域，许多潜在的偏见来源已被确定在周围的话语， 强大的先验信息来源，以缓和不确定性，但它们对个人估计的影响， 证据库的关联和摘要尚未完全量化。数量调整 这些错误使用定量偏倚分析和贝叶斯方法-以及荟萃分析的发表偏倚 总结-将提高再现性。然后，我们将扩展和应用网络信息学工具， 实施方法的任何主题与一组异构的研究结果，让利益相关者没有 先进的分析技能，以调整基本假设，并看到自己的影响 总结结果。通过实现我们的目标，该项目将促进研究信息学的使用， 再现性危机，有助于加速对任何研究课题建立共识，并有效地引导 研究资源，以解决任何主题领域中最具影响力的不确定性来源。

项目成果

Yland et al. Respond to "Heuristics and Wish Bias".

伊兰等人。

• DOI: 10.1093/aje/kwac092
• 发表时间: 2022
• 期刊: American journal of epidemiology
• 影响因子: 5
• 作者: Yland,JenniferJ;Wesselink,AmeliaK;Lash,TimothyL;Fox,MatthewP
• 通讯作者: Fox,MatthewP

SAS and R code for probabilistic quantitative bias analysis for misclassified binary variables and binary unmeasured confounders.

用于错误分类二元变量和二元未测量混杂因素的概率定量偏差分析的 SAS 和 R 代码。

• DOI: 10.1093/ije/dyad053
• 发表时间: 2023
• 期刊: International journal of epidemiology
• 影响因子: 7.7
• 作者: Fox,MatthewP;MacLehose,RichardF;Lash,TimothyL
• 通讯作者: Lash,TimothyL

Application of the Adaptive Validation Substudy Design to Colorectal Cancer Recurrence.

适应性验证子研究设计在结直肠癌复发中的应用。

• DOI: 10.2147/clep.s230314
• 发表时间: 2020
• 期刊: Clinical epidemiology
• 影响因子: 3.9
• 作者: Collin,LindsayJ;Riis,AndersH;MacLehose,RichardF;Ahern,ThomasP;Erichsen,Rune;Thorlacius-Ussing,Ole;Lash,TimothyL
• 通讯作者: Lash,TimothyL

Practical data considerations for the modern epidemiology student.

现代流行病学学生的实用数据注意事项。

• DOI: 10.1016/j.gloepi.2021.100066
• 发表时间: 2021
• 期刊: Global epidemiology
• 作者: Tran,NguyenK;Lash,TimothyL;Goldstein,NealD
• 通讯作者: Goldstein,NealD

Opioid-related Mortality in United States Death Certificate Data: A Quantitative Bias Analysis With Expert Elicitation of Bias Parameters.

美国死亡证明数据中与阿片类药物相关的死亡率：专家提取偏差参数的定量偏差分析。

• DOI: 10.1097/ede.0000000000001600
• 发表时间: 2023
• 期刊: Epidemiology (Cambridge, Mass.)
• 作者: Goldsmith,ElizabethS;Krebs,ErinE;Ramirez,MarizenR;MacLehose,RichardF
• 通讯作者: MacLehose,RichardF