Optimizing Risk-Adjustment for Measuring ICU Quality

这一职业发展奖的目的是开发一个研究和培训计划，使Glance博士能够成为健康结果研究领域的独立医生调查员。这项提案的重点将是研究在大型多机构数据库中使用风险调整模型作为重症监护病房(ICU)绩效基准的可行性。粗死亡率不能用来比较ICU，因为它们没有考虑到ICU患者群体之间疾病严重程度的差异。然而，标准化死亡率比率(SMR)，即观察死亡率(OMR)与预期死亡率(EMR)的比率，确实考虑到了ICU之间患者病例组合的差异，因此可以用来比较各个ICU的表现。EMR是通过使用ICU评分系统计算特定人群中患者的预测死亡率而获得的。使用严重程度调整的结果衡量标准，如SMR，可以识别低性能和高性能ICU。这种方法的一个潜在问题是，发现一般的严重性评分系统在如何使用严重性调整后的结果衡量标准对个别医院进行排名方面存在分歧。ICU评分系统很可能会以同样的方式运行；具体目标#1将针对APACHE 11(急性生理学和慢性健康评估)、SAPS 11(简化急性生理学评分)和MPM II(死亡率概率模型)检验这一假设。发现不同的评分系统在质量异常值的身份上存在分歧，这一发现引发了人们对使用严重程度调整的结果衡量标准的努力表面有效性的怀疑。验证研究表明，ICU评分系统在大多数数据集(除了用于开发原始模型的数据集)中进行评估时表现不佳。因此，当其所基于的评分系统不准确时，严重程度调整后的结果衡量标准在医院质量上不一致也就不足为奇了。然而，预测模型可以根据数据集进行定制，以便它们可以更准确地预测该数据集中的结果。具体目标#2将审查这一命题，即ICU评分系统在根据ICU数据库定制评分系统后，更有可能就质量异常值的身份达成一致，ICU数据库由其性能被作为基准的ICU组成。

The purpose of this career development award is to develop a research and training program that will enable Dr. Glance to become an independent physician investigator in the field of health outcomes research. The focus of this proposal will be to examine the feasibility of using risk-adjustment models for benchmarking Intensive Care Unit (ICU) performance in a large multi-institutional database. Crude mortality rates cannot be used to compare ICUs because they do not account for differences in the severity of disease between ICU patient populations. However, the Standardized Mortality Ratio (SMR), defined as the ratio of the observed mortality rate (OMR) to the expected mortality rate (EMR), does account for differences in patient case mix between ICUs, and can therefore be used to compare the performance of individual ICUs. The EMR is obtained by calculating the predicted mortality of the patients in a specific population using an ICU scoring system. Using severity-adjusted outcome measures like the SMR, low- and high-performance ICUs can be identified. A potential problem with this approach is the finding that general severity scoring systems disagree on how to rank individual hospitals using severity-adjusted outcome measures. It is likely that ICU scoring systems will behave in the same manner; Specific Aim #1 will examine this hypothesis for APACHE 11 (Acute Physiology and Chronic Health Evaluation), SAPS 11 (Simplified Acute Physiology Score) and MPM II (Mortality Probability Model). A finding that different scoring systems disagree on the identity of quality outliers raises doubt regarding the face validity of benchmarking efforts using severity-adjusted outcome measures. Validation studies have demonstrated that ICU scoring systems perform poorly when evaluated in most data sets (other than the one used to develop the original model). Therefore, it not surprising that severity-adjusted outcome measures do not agree on hospital quality when the scoring systems on which they are based are not accurate. However, predictive models can be customized to a data set so that they will more accurately predict outcomes within that data set. Specific Aim #2 will examine the proposition that ICU scoring systems are more likely to agree on the identity of quality outliers after the scoring systems have been customized to the ICU database made up of the ICUs whose performance is being benchmarked.