Generalization in Clinical Prediction Models: The Blessing and Curse of Measurement Indicator Variables.
Generalization in Clinical Prediction Models: The Blessing and Curse of Measurement Indicator Variables.
复制标题
DOI：
10.1097/cce.0000000000000453
复制
发表时间：
2021-07
影响因子：
--
通讯作者：
Kamaleswaran R
中科院分区：
文献类型：
--
作者：
Futoma J;Simons M;Doshi-Velez F;Kamaleswaran R
补充数字内容可在文本中找到。影响临床预测模型普遍性的具体因素知之甚少。我们的主要目的是调查测量指标变量如何影响临床预测模型的外部效度，以预测血管加压药治疗的开始。我们使用两类变量对回顾性队列进行logistic回归分析，以预测血管加压药的起效：看似客观的临床变量（生命体征和实验室测量）和表示测量结果近期性的更主观的变量。三个队列来自地理上不同地区的两个三级护理学术医院，跨越普通住院和重症监护环境。每个队列均由成人患者（住院时年龄大于或等于18岁）组成，住院时间为6至600小时，并且在住院或ICU入院的前6小时内未接受血管加压药。在三个衍生队列中的每一个上开发模型，并在衍生队列上进行内部验证，在其他两个队列上进行外部验证。没有。血管加压药的使用率在普通住院队列中为0.9%，在两个重症监护队列中分别为12.4%和11.5%。利用两类变量的模型在样本内表现最佳，预测4小时内血管加压药起效的C统计量分别为0.862（95% CI，0.844-0.879）、0.822（95% CI，0.793-0.852）和0.889（95% CI，0.880-0.898）。仅使用主观变量表示测量近因的模型外部效度较差。然而，这些实践驱动的变量有助于调整两家医院之间的差异，并导致更普遍的模型使用临床变量。我们开发和外部验证模型用于预测血管加压药的起效。我们发现，特定于实践的功能，表示测量的新近度提高了本地性能，也导致了更普遍的模型，如果他们在模型开发过程中进行调整，但在验证时丢弃。如果目标是开发可推广的模型，则应仔细考虑临床预测建模中特定于实践的特征（如测量指标）的作用。
Supplemental Digital Content is available in the text. Specific factors affecting generalizability of clinical prediction models are poorly understood. Our main objective was to investigate how measurement indicator variables affect external validity in clinical prediction models for predicting onset of vasopressor therapy. We fit logistic regressions on retrospective cohorts to predict vasopressor onset using two classes of variables: seemingly objective clinical variables (vital signs and laboratory measurements) and more subjective variables denoting recency of measurements. Three cohorts from two tertiary-care academic hospitals in geographically distinct regions, spanning general inpatient and critical care settings. Each cohort consisted of adult patients (age greater than or equal to 18 yr at time of hospitalization), with lengths of stay between 6 and 600 hours, and who did not receive vasopressors in the first 6 hours of hospitalization or ICU admission. Models were developed on each of the three derivation cohorts and validated internally on the derivation cohort and externally on the other two cohorts. None. The prevalence of vasopressors was 0.9% in the general inpatient cohort and 12.4% and 11.5% in the two critical care cohorts. Models utilizing both classes of variables performed the best in-sample, with C-statistics for predicting vasopressor onset in 4 hours of 0.862 (95% CI, 0.844–0.879), 0.822 (95% CI, 0.793–0.852), and 0.889 (95% CI, 0.880–0.898). Models solely using the subjective variables denoting measurement recency had poor external validity. However, these practice-driven variables helped adjust for differences between the two hospitals and led to more generalizable models using clinical variables. We developed and externally validated models for predicting the onset of vasopressors. We found that practice-specific features denoting measurement recency improved local performance and also led to more generalizable models if they are adjusted for during model development but discarded at validation. The role of practice-specific features such as measurement indicators in clinical prediction modeling should be carefully considered if the goal is to develop generalizable models.