A standardized analytics pipeline for reliable and rapid development and validation of prediction models using observational health data.
A standardized analytics pipeline for reliable and rapid development and validation of prediction models using observational health data.
复制标题
DOI：
10.1016/j.cmpb.2021.106394
复制
发表时间：
2021-11
影响因子：
6.1
通讯作者：
Reps JM
中科院分区：
文献类型：
--
作者：
Khalid S;Yang C;Blacketer C;Duarte-Salles T;Fernández-Bertolín S;Kim C;Park RW;Park J;Schuemie MJ;Sena AG;Suchard MA;You SC;Rijnbeek PR;Reps JM
为应对持续的COVID-19疫情，现有文献中的多个预测模型迅速发展，旨在提供循证指导。然而，这些COVID-19预测模型均不可靠。模型通常被评估为具有偏倚风险，通常是由于报告不足，使用非代表性数据以及缺乏大规模外部验证。在本文中，我们提出了用于患者级预测建模的观察性健康数据科学和信息学（OHDSI）分析管道，作为快速可靠开发和验证预测模型的标准化方法。我们展示了我们的分析管道和开源软件工具如何用于回答重要的预测问题，同时限制潜在的偏差原因（例如，通过验证表型、指定目标人群、进行大规模外部验证以及公开提供所有分析源代码）。我们逐步展示了如何针对以下问题实施分析流程：“在因COVID-19住院的患者中，住院后0至30天的死亡风险是多少？”。我们在美国索赔数据库中使用六种不同的机器学习方法开发模型，该数据库包含超过20，000例COVID-19住院病例，并使用来自韩国，西班牙和美国的超过45，000例COVID-19住院病例的数据对模型进行外部验证。我们的开源软件工具使我们能够有效地从问题设计到可靠的模型开发和评估。在预测COVID-19住院患者的死亡时，AdaBoost、随机森林、梯度增强机和决策树产生了与L1正则化逻辑回归相似或更低的内部和外部验证区分性能，而MLP神经网络始终导致较低的区分。L1-正则化逻辑回归模型被很好地校准。我们的研究结果表明，遵循OHDSI分析管道进行患者级预测建模可以快速开发可靠的预测模型。OHDSI软件工具和管道是开源的，可供来自世界各地的研究人员使用。
As a response to the ongoing COVID-19 pandemic, several prediction models in the existing literature were rapidly developed, with the aim of providing evidence-based guidance. However, none of these COVID-19 prediction models have been found to be reliable. Models are commonly assessed to have a risk of bias, often due to insufficient reporting, use of non-representative data, and lack of large-scale external validation. In this paper, we present the Observational Health Data Sciences and Informatics (OHDSI) analytics pipeline for patient-level prediction modeling as a standardized approach for rapid yet reliable development and validation of prediction models. We demonstrate how our analytics pipeline and open-source software tools can be used to answer important prediction questions while limiting potential causes of bias (e.g., by validating phenotypes, specifying the target population, performing large-scale external validation, and publicly providing all analytical source code). We show step-by-step how to implement the analytics pipeline for the question: ‘In patients hospitalized with COVID-19, what is the risk of death 0 to 30 days after hospitalization?’. We develop models using six different machine learning methods in a USA claims database containing over 20,000 COVID-19 hospitalizations and externally validate the models using data containing over 45,000 COVID-19 hospitalizations from South Korea, Spain, and the USA. Our open-source software tools enabled us to efficiently go end-to-end from problem design to reliable Model Development and evaluation. When predicting death in patients hospitalized with COVID-19, AdaBoost, random forest, gradient boosting machine, and decision tree yielded similar or lower internal and external validation discrimination performance compared to L1-regularized logistic regression, whereas the MLP neural network consistently resulted in lower discrimination. L1-regularized logistic regression models were well calibrated. Our results show that following the OHDSI analytics pipeline for patient-level prediction modelling can enable the rapid development towards reliable prediction models. The OHDSI software tools and pipeline are open source and available to researchers from all around the world.
登录
查看更多内容
在COVID-19患者中使用重新利用和辅助药物：跨国网络队列研究。
DOI：
10.1136/bmj.n1038
发表时间：
2021-05-11
期刊：
BMJ (Clinical research ed.)
影响因子：
--
作者：
Prats-Uribe A;Sena AG;Lai LYH;Ahmed WU;Alghoul H;Alser O;Alshammari TM;Areia C;Carter W;Casajust P;Dawoud D;Golozar A;Jonnagaddala J;Mehta PP;Gong M;Morales DR;Nyberg F;Posada JD;Recalde M;Roel E;Shah K;Shah NH;Schilling LM;Subbian V;Vizcaya D;Zhang L;Zhang Y;Zhu H;Liu L;Cho J;Lynch KE;Matheny ME;You SC;Rijnbeek PR;Hripcsak G;Lane JC;Burn E;Reich C;Suchard MA;Duarte-Salles T;Kostka K;Ryan PB;Prieto-Alhambra D
通讯作者：
Prieto-Alhambra D
DOI：
10.1038/s41366-021-00893-4
发表时间：
2021-11
期刊：
International journal of obesity (2005)
影响因子：
--
作者：
Recalde M;Roel E;Pistillo A;Sena AG;Prats-Uribe A;Ahmed WU;Alghoul H;Alshammari TM;Alser O;Areia C;Burn E;Casajust P;Dawoud D;DuVall SL;Falconer T;Fernández-Bertolín S;Golozar A;Gong M;Lai LYH;Lane JCE;Lynch KE;Matheny ME;Mehta PP;Morales DR;Natarjan K;Nyberg F;Posada JD;Reich CG;Rijnbeek PR;Schilling LM;Shah K;Shah NH;Subbian V;Zhang L;Zhu H;Ryan P;Prieto-Alhambra D;Kostka K;Duarte-Salles T
通讯作者：
Duarte-Salles T
影响因子：
16.6
作者：
Burn E;You SC;Sena AG;Kostka K;Abedtash H;Abrahão MTF;Alberga A;Alghoul H;Alser O;Alshammari TM;Aragon M;Areia C;Banda JM;Cho J;Culhane AC;Davydov A;DeFalco FJ;Duarte-Salles T;DuVall S;Falconer T;Fernandez-Bertolin S;Gao W;Golozar A;Hardin J;Hripcsak G;Huser V;Jeon H;Jing Y;Jung CY;Kaas-Hansen BS;Kaduk D;Kent S;Kim Y;Kolovos S;Lane JCE;Lee H;Lynch KE;Makadia R;Matheny ME;Mehta PP;Morales DR;Natarajan K;Nyberg F;Ostropolets A;Park RW;Park J;Posada JD;Prats-Uribe A;Rao G;Reich C;Rho Y;Rijnbeek P;Schilling LM;Schuemie M;Shah NH;Shoaibi A;Song S;Spotnitz M;Suchard MA;Swerdel JN;Vizcaya D;Volpe S;Wen H;Williams AE;Yimer BB;Zhang L;Zhuk O;Prieto-Alhambra D;Ryan P
通讯作者：
Ryan P
DOI：
10.1016/s2589-7500(20)30289-2
发表时间：
2021-03
期刊：
The Lancet. Digital health
影响因子：
--
作者：
Morales DR;Conover MM;You SC;Pratt N;Kostka K;Duarte-Salles T;Fernández-Bertolín S;Aragón M;DuVall SL;Lynch K;Falconer T;van Bochove K;Sung C;Matheny ME;Lambert CG;Nyberg F;Alshammari TM;Williams AE;Park RW;Weaver J;Sena AG;Schuemie MJ;Rijnbeek PR;Williams RD;Lane JCE;Prats-Uribe A;Zhang L;Areia C;Krumholz HM;Prieto-Alhambra D;Ryan PB;Hripcsak G;Suchard MA
通讯作者：
Suchard MA
DOI：
10.1093/jamia/ocy032
发表时间：
2018-08-01
影响因子：
6.4
作者：
Reps, Jenna M.;Schuemie, Martijn J.;Rijnbeek, Peter R.
通讯作者：
Rijnbeek, Peter R.