How robust are hospital ranks based on composite performance measures?
How robust are hospital ranks based on composite performance measures?
复制标题
DOI：
10.1097/01.mlr.0000185692.72905.4a
复制
发表时间：
2005-12-01
期刊：
影响因子：
3
通讯作者：
Smith, PC
中科院分区：
文献类型：
--
作者：
Jacobs, R;Goddard, M;Smith, PC
背景：医疗保健绩效的综合指数是潜在的个人绩效指标的集合。它们越来越多地被用于对医疗机构进行排名。虽然综合指标以一种易于解释的方式呈现“大局”，但如果不注意其构建过程中的关键方法问题，可能会得出误导性结论。目的：我们在构建和使用复合测量的背景下检查性能测量的可变性。我们说明了基础数据的可变性和由此产生的组合可能会破坏医疗保健绩效指标的稳健性。我们还说明了用于汇总单个指标的方法规则的变化如何对综合分数产生重要影响。方法：我们使用117家英国急症医院的数据来说明一般的方法学问题。绩效度量的差异被划分为“可控”和“不可控”的因素。我们从潜在的性能指标创建一个综合指数，并使用蒙特卡罗模拟来检查综合的鲁棒性。结果：超出组织控制的随机变异导致医院评分存在相当大的不确定性。组合对权重系统和聚合规则的更改也很敏感。由于一些细微的变化，一些医院的排名可以跃升近一半。结论：在解释综合性能测量结果时需要非常小心。并对其未来发展提出了建议。
Background: Composite indices of healthcare performance are an aggregation of underlying individual performance measures. They are increasingly being used to rank healthcare organizations. Although composite indicators present the "big picture" in a way that is easy to interpret, misleading conclusions may be drawn if attention is not paid to key methodological issues in their construction.Objectives: We examine variability in performance measures in the context of the construction and use of composite measures. We illustrate how variability in the underlying data and the resulting composite may undermine the robustness of performance measures in health care. We also illustrate how variation in the methodological rules applied to aggregate the individual indicators can have an important impact on composite scores.Methods: We use data for 117 English acute hospitals to illustrate the generic methodological issues. The variance in performance measures is partitioned into "controllable" and "uncontrollable" elements. We create a composite index from the underlying performance indicators and use Monte Carlo simulations to examine the robustness of the composite.Results: Random variation beyond the control of organizations gives rise to considerable uncertainty in hospital scores. Composites are also sensitive to changes made to the weighting system and to the aggregation rules. Some hospitals can jump almost half of the league table as a result of subtle changes.Conclusions: Great care is warranted in interpreting the results of composite performance measures. Suggestions for their future development are made.