Using public clinical trial reports to probe non-experimental causal inference methods.
Using public clinical trial reports to probe non-experimental causal inference methods.
复制标题
DOI：
10.1186/s12874-023-02025-0
复制
发表时间：
2023-09-09
影响因子：
4
通讯作者：
中科院分区：
文献类型：
--
作者：
非实验性研究（也称为观察性研究）对于估计各种医疗干预措施的效果很有价值，但众所周知，难以评估，因为非实验性研究中使用的方法需要不可验证的假设。这种内在可验证性的缺乏使得人们很难比较不同的非实验性研究方法，也很难相信任何特定的非实验性研究的结果。我们介绍TrialProbe，一个用于评估非实验方法的数据资源和统计框架。我们首先收集了一个数据集的伪“地面真理”的相对影响的药物，通过使用经验贝叶斯技术来分析不良事件记录在公共临床试验报告。然后，我们开发了一个框架，通过测量非实验性效果估计值和来自临床试验的估计值之间的一致性，来评估非实验性方法对地面真相的影响。作为我们的方法的演示，我们还执行了一个示例方法之间的评价倾向得分匹配，逆倾向得分加权，和一个大型的国家保险索赔数据集上的未调整的方法。从我们的ClinicalTrials.gov数据集中的33，701个临床试验记录中，我们能够提取12，967个独特的药物/药物不良事件比较，以形成一个基础事实集。在我们相应的方法评估过程中，我们能够使用该参考集来证明倾向评分匹配和反向倾向评分加权都可以产生与临床试验结果高度一致的估计值，并且大大优于未经调整的基线。我们发现，TrialProbe是一种有效的方法来探测非实验性的研究方法，能够生成大的地面真值集，能够区分非实验性方法在真实的世界观测数据中的表现。
Non-experimental studies (also known as observational studies) are valuable for estimating the effects of various medical interventions, but are notoriously difficult to evaluate because the methods used in non-experimental studies require untestable assumptions. This lack of intrinsic verifiability makes it difficult both to compare different non-experimental study methods and to trust the results of any particular non-experimental study. We introduce TrialProbe, a data resource and statistical framework for the evaluation of non-experimental methods. We first collect a dataset of pseudo “ground truths” about the relative effects of drugs by using empirical Bayesian techniques to analyze adverse events recorded in public clinical trial reports. We then develop a framework for evaluating non-experimental methods against that ground truth by measuring concordance between the non-experimental effect estimates and the estimates derived from clinical trials. As a demonstration of our approach, we also perform an example methods evaluation between propensity score matching, inverse propensity score weighting, and an unadjusted approach on a large national insurance claims dataset. From the 33,701 clinical trial records in our version of the ClinicalTrials.gov dataset, we are able to extract 12,967 unique drug/drug adverse event comparisons to form a ground truth set. During our corresponding methods evaluation, we are able to use that reference set to demonstrate that both propensity score matching and inverse propensity score weighting can produce estimates that have high concordance with clinical trial results and substantially outperform an unadjusted baseline. We find that TrialProbe is an effective approach for probing non-experimental study methods, being able to generate large ground truth sets that are able to distinguish how well non-experimental methods perform in real world observational data.
登录
查看更多内容
影响因子：
168.9
作者：
DeVito, Nicholas J.;Bacon, Seb;Goldacre, Ben
通讯作者：
Goldacre, Ben
影响因子：
5.7
作者：
Dorie, Vincent;Hill, Jennifer;Cervone, Dan
通讯作者：
Cervone, Dan
影响因子：
6.3
作者：
Athey, Susan;Imbens, Guido W.;Munro, Evan
通讯作者：
Munro, Evan
影响因子：
5
作者：
Hernan, Miguel A.;Robins, James M.
通讯作者：
Robins, James M.
影响因子：
3.7
作者：
Efron, B
通讯作者：
Efron, B