The impact of different prognostic models and their customization on institutional comparison of intensive care units
The impact of different prognostic models and their customization on institutional comparison of intensive care units
复制标题
DOI：
10.1097/01.ccm.0000288123.29559.5a
复制
发表时间：
2007-11-01
影响因子：
8.8
通讯作者：
de Keizer, Nicolette F.
中科院分区：
文献类型：
--
作者：
Bakhshi-Raiez, Ferishta;Peek, Niels;de Keizer, Nicolette F.
目标。目的：评估预后模型选择的影响以及定制这些模型对重症监护病房(ICU)排序表(即，排名列表)的影响，在排序表中，重症监护病房(ICU)根据标准化死亡率使用急性生理学和慢性健康评估(APACHE)11、简化急性生理学评分(SAPS)11和死亡概率模型11(MPM，11)进行排名。对前瞻性收集的ICU入院数据进行回顾分析。设置。40个荷兰ICU。患者：2002年1月至2006年10月来自国家登记的86,427名患者的数据集。比较了与不同模型相关的排名表，以评估它们的一致性。采用自助法对ICU等级中的不确定性进行量化。首先，对于每个ICU，确定每个模型的中位数等级及其95%的可信区间。然后，对于给定的模型对，对于每个ICU，计算排名的中位数差异及其相关的95%可信区间。对于一对给定的模型，ICU的排名差异被认为是相关的，如果它具有统计学意义，如果其中一个模型将该ICU归类为性能异常值(表现优秀或非常差)，而另一个不是。测量和主要结果。对于20个ICU，一对或多对模型之间的排名(2-19个位置)有显著差异。三个ICU被其中一个模型评为性能异常值，而另一个模型以95%的确定性排除了这种可能性。此外，对于十个ICU，一对或多对模型将这些ICU归类为性能异常值，而另一对模型并不确定地将其归类为性能异常值。关于原始模型和定制模型之间的一致性，在所有情况下，排名变化的中位数都在3个位置或更少，并且模型完全同意哪些ICU应该被归类为性能异常值。基于病例组合调整排名表的机构比较对预测模型的选择敏感，而对这些模型的定制不敏感。排行榜应该始终显示与机构排名相关的不确定性。
Objectives. To evaluate the influence of choice of a prognostic model and the effect of customization of these models on league tables (i.e., rank-order listing) in which intensive care units (ICUs) are ranked by standardized mortality ratios using Acute Physiology and Chronic Health Evaluation (APACHE) 11, Simplified Acute Physiology Score (SAPS) 11, and Mortality Probability Model 11 (MPM,,11).Design. Retrospective analysis of prospectively collected data on ICU admissions. Setting. Forty Dutch ICUs.Patients: A data set from a national registry of 86,427 patients from January 2002 to October 2006.Interventions. The league tables associated with the different models were compared to evaluate their agreement. Bootstrapping was used to quantify the uncertainty in the ranks for ICUs. First, for each ICU the median rank and its 95% confidence interval were identified for each model. Then, for a given pair of models, for each ICU the median difference in rank and its associated 95% confidence interval were computed. A difference in rank for an ICU for a given pair of models was considered relevant if it was statistically significant and if one of the models would categorize this ICU as a performance outlier (excellent performer or very poor performer) while the other did not.Measurements and Main Results. For 20 ICUs, there was a significant difference in rank (2-19 positions) between one or more-pairs of models. Three ICUs were rated as performance outliers by one of the models, while the other excluded this possibility with 95% certainty. Furthermore, for ten ICUs, one or more pairs of models classified these ICUs as performance outliers while the other model did not do so with certainty. Regarding the agreement between the original models and their customized versions, in all cases the median change in rank was three positions or less and the models fully agreed with respect to which ICUs should be classified as performance outliers.Conclusions. Institutional comparison based on case-mix adjusted league tables is sensitive to the choice of prognostic model but not to customization of these models. League tables should always display the uncertainty associated with institutional ranks.