Benchmarking Hospital Quality: Template Matching versus Conventional Regression Approaches

背景：识别和补救低质量护理是系统质量改进的核心， 尤其是在退伍军人事务部。然而，跨医院比较以确定表现不佳的医院受到以下限制 患者病例组合和病情严重程度的差异。临床医生考虑当前的基准方法 用传统的回归是不公平的、不清楚的和无济于事的。虽然大量资源被投入到 医院标杆，目前这项投资的回报是有限的，因为临床医生不理解或 相信这些方法。美国国家医学科学院最近呼吁投资于科学研究 业绩衡量，并提高其透明度和有效性。模板匹配已经完成 提出了一种公平、清晰和有帮助的替代基准方法。然而， 这一新方法从未在有限的研究环境外进行过测试。 具体目标：测试模板匹配在比较退伍军人管理局不同急性白血病患者护理质量方面的效用 该项目将评估这种方法的可行性、准确性和可解释性。 具体地说，该项目将：(A1)可行性：开发和优化两种模板匹配方法 比较退伍军人医院的30天死亡率。(A2)准确率：比较模板匹配能力 而不是用传统的回归来正确识别表现不佳的医院。(A3)可解释性：比较 模板匹配生成的医院绩效数据的可解释性和可信度 具有临床领导者的传统回归模型。 预期影响：这项提案的首要目标是改善住院退伍军人的护理。 该提案将把前景看好的新模板匹配方法应用于不同的退伍军人管理局 并完成对这些方法的多方面评估。我们不仅会考虑 模板匹配的可行性和方法严谨性，以及可解释性、可信度和 数据的问责制。我们预计，模板匹配将是可行的基准VA急性护理 医院，它将至少识别表现不佳的医院以及当前的基准 传统回归，对退伍军人管理局局长来说，它将比 当前业绩报告。我们预计这项工作未来会有很多扩展，包括模板的使用 将退伍军人事务部与私营部门进行比较。 特色和创新：这项建议是创新的，因为它将同时(1)推进 (2)在咨询中发展必要的基础设施 与运营合作伙伴在退伍军人管理局中使用模板匹配，以及(3)不仅评估这一新方法 从统计上讲，但也要考虑到它应该告知的实际用户。结果与退伍军人管理局高度相关，其 业绩总是受到密切关注。在退伍军人管理局之外，它也引起了人们的极大兴趣，因为许多医疗保健系统 世卫组织同样努力为个别医院的业绩提供有意义和可操作的评估。 项目方法：目标1将制定和完善30天死亡率基准的统计方法 在VA急性护理医院中使用两种模板匹配方法：(1)单一模板和(2)个性化 每家医院的模板。将考虑多种统计匹配方法，以实现最公平的匹配 不同医院之间的比较。目标2将使用模拟和真实患者数据来衡量时间和原因 模板匹配和传统的回归方法可能会对医院产生不一致的评估。 目标3将制定模板匹配数据的呈现方式，然后调查退伍军人管理局医务长进行比较 使用模板匹配与常规匹配的业绩报告的可解释性和可信度 对医院基准进行回归，并对部分首席执行官进行半结构化随访，以 从医学科长的角度进一步理解模板匹配绩效报告。

Background: Identifying and remediating low-quality care is at the heart of systematic quality improvement, particularly in VA. However, cross-hospital comparisons to identify under-performing hospitals are limited by differences in patient case-mix and illness severity. Clinicians consider the current approach to benchmarking with conventional regression to be unfair, unclear, and unhelpful. While substantial resources are devoted to hospital benchmarking, the current return on this investment is limited because clinicians do not understand or trust the methods. The National Academy of Medicine has recently called for investing into the science of performance measurement and for increasing its transparency and validity. Template matching has been proposed as an alternative methodological approach to benchmarking that is fair, clear, and helpful. However, this new approach has never been tested outside of limited research settings. Specific Aims: To test the utility of template matching for comparing quality of care across VA's diverse acute care hospitals, this project will assess the feasibility, accuracy, and interpretability of this approach. Specifically, the project will: (A1) Feasibility: Develop and optimize two template matching approaches for comparing 30-day mortality across VA hospitals. (A2) Accuracy: Compare the ability of template matching versus conventional regression to correctly identify under-performing hospitals. (A3) Interpretability: Compare the interpretability and credibility of hospital performance data generated from template matching versus conventional regression models with clinical leaders. Anticipated Impact: The overarching goal of this proposal is to improve the care of hospitalized Veterans. This proposal will apply promising, new template matching approaches for benchmarking to the diverse VA healthcare system and complete a multi-faceted evaluation of these approaches. We will consider not only the feasibility and methodological rigor of template matching, but also the interpretability, credibility and accountability of the data. We expect that template matching will be feasible for benchmarking VA acute care hospitals, that it will identify under-performing hospitals at least as well as current benchmarking with conventional regression, and that it will be more interpretable and credible to VA Chiefs of Medicine than current performance reports. We anticipate many future expansions to this work, including the use of template matching for VA to private sector comparisons. Unique Features and Innovation: This proposal is innovative because it will simultaneously (1) advance the statistical state-of-the-art in hospital benchmarking, (2) develop the necessary infrastructure in consultation with operational partners to use template matching in VA, and (3) evaluate this new approach not just statistically, but also with the actual users it is supposed to inform. The results are highly relevant to VA, whose performance is always under scrutiny. It is also of great interest outside VA, for the many healthcare systems who similarly struggle to provide meaningful and actionable assessments of individual hospital's performance. Project Methods: Aim 1 will develop and refine the statistical methodology for benchmarking 30-day mortality in VA acute care hospitals using two template matching approaches: (1) a single template and (2) personalized templates for each hospital. Multiple statistical approaches to matching will be considered to achieve the fairest comparisons across hospitals. Aim 2 will use simulation and real patient data to measure when and why template matching and conventional regression approaches may yield discordant assessments of hospitals. Aim 3 will develop the presentation of template matching data, then survey VA Chiefs of Medicine to compare the interpretability and credibility of performance reports using template matching versus conventional regression for hospital benchmarking, with follow-up semi-structured interviews with a subset of Chiefs to further understand template matching performance reports from the perspective of Chiefs of Medicine.