Design and implementation of a standardized framework to generate and evaluate patient-level prediction models using observational healthcare data
Design and implementation of a standardized framework to generate and evaluate patient-level prediction models using observational healthcare data
复制标题
DOI：
10.1093/jamia/ocy032
复制
发表时间：
2018-08-01
影响因子：
6.4
通讯作者：
Rijnbeek, Peter R.
中科院分区：
文献类型：
--
作者：
Reps, Jenna M.;Schuemie, Martijn J.;Rijnbeek, Peter R.
目的：为了开发一个包含标准化步骤的概念预测模型框架，并描述相应的开源软件开发，以一致地实现跨计算环境和观察医疗数据库的框架，使模型共享和reproducibility.Methods：基于现有的最佳实践，我们提出了一个5步标准化框架：（1）透明地定义问题;（2）选择合适的数据集;（3）从观测数据构造变量;（4）学习预测模型;以及（5）验证模型性能。我们将此框架作为开源软件实施，利用观察性医学成果伙伴关系通用数据模型，方便共享模型并在多个观察数据集上复制模型评估。软件实现包含默认的协变量和分类器，但框架，使定制和extension.Results：作为一个概念验证，展示的透明度和易用性的模型传播使用的软件，我们开发了预测模型的21个不同的结果在目标人群中的人患有抑郁症在4个观测数据库。所有84个模型都可以在一个可访问的在线存储库中实现，任何人都可以访问公共数据模型格式的观测数据库。结论：概念验证研究说明了该框架的能力，开发可重复的模型，可以很容易地共享，并提供了潜在的执行广泛的外部验证模型，并提高其临床吸收的可能性。在未来的工作中，该框架将被应用于执行“全面”预测分析，以评估众多目标人群、结果和时间以及风险设置的观测数据预测域。
Objective: To develop a conceptual prediction model framework containing standardized steps and describe the corresponding open-source software developed to consistently implement the framework across computational environments and observational healthcare databases to enable model sharing and reproducibility.Methods: Based on existing best practices we propose a 5 step standardized framework for: (1) transparently defining the problem; (2) selecting suitable datasets; (3) constructing variables from the observational data; (4) learning the predictive model; and (5) validating the model performance. We implemented this framework as open-source software utilizing the Observational Medical Outcomes Partnership Common Data Model to enable convenient sharing of models and reproduction of model evaluation across multiple observational datasets. The software implementation contains default covariates and classifiers but the framework enables customization and extension.Results: As a proof-of-concept, demonstrating the transparency and ease of model dissemination using the software, we developed prediction models for 21 different outcomes within a target population of people suffering from depression across 4 observational databases. All 84 models are available in an accessible online repository to be implemented by anyone with access to an observational database in the Common Data Model format.Conclusions: The proof-of-concept study illustrates the framework's ability to develop reproducible models that can be readily shared and offers the potential to perform extensive external validation of models, and improve their likelihood of clinical uptake. In future work the framework will be applied to perform an "all-by-all" prediction analysis to assess the observational data prediction domain across numerous target populations, outcomes and time, and risk settings.