Benchmarking Hospital Quality: Template Matching versus Conventional Regression Approaches

背景：识别和补救低质量的护理是系统质量改进的核心， 尤其是在VA。然而，跨医院比较，以确定表现不佳的医院是有限的， 患者病例组合和疾病严重程度的差异。临床医生认为目前的基准方法 传统的回归是不公平的，不清楚的，没有帮助的。虽然大量资源用于 医院基准，目前这项投资的回报是有限的，因为临床医生不了解或 相信方法。美国国家医学院最近呼吁投资于 业绩衡量，并提高其透明度和有效性。模板匹配已被 提出了一种公平、清晰和有用的基准测试的替代方法。然而，在这方面， 这种新方法从未在有限的研究环境之外进行过测试。 具体目的：测试模板匹配在比较VA不同急性期护理质量方面的效用。 护理医院，该项目将评估这种方法的可行性，准确性和可解释性。 （A1）可行性：开发和优化两种模板匹配方法， 比较退伍军人医院的30天死亡率(A2)准确性：比较模板匹配的能力 与传统回归相比，可以正确识别表现不佳的医院。(A3)解释性：比较 从模板匹配生成的医院绩效数据的可解释性和可信度， 传统的回归模型。 预期影响：该提案的总体目标是改善住院退伍军人的护理。 这项建议将适用于有前途的，新的模板匹配方法的基准，以不同的VA 医疗保健系统，并完成这些方法的多方面评估。我们不仅要考虑 模板匹配的可行性和方法的严谨性，而且可解释性，可信度和 数据的责任。我们希望模板匹配将是可行的基准VA急性护理 医院，它将确定表现不佳的医院，至少以及目前的基准， 传统的回归，它将是更可解释和可信的VA医学主任比 目前的业绩报告。我们预计许多未来的扩展这项工作，包括使用模板 将VA与私营部门进行比较。 特色与创新：该提案具有创新性，因为它将同时（1）推进 医院基准的统计学最新水平，（2）在协商中发展必要的基础设施 与运营合作伙伴一起在VA中使用模板匹配，以及（3）评估这种新方法， 从统计上讲，也与它应该告知的实际用户有关。结果与VA高度相关，其 业绩始终受到审查。在VA之外，对于许多医疗保健系统来说， 他们同样努力为个别医院的表现提供有意义和可操作的评估。 项目方法：目标1将制定和完善30天死亡率基准的统计方法 在VA急性护理医院中使用两种模板匹配方法：（1）单个模板和（2）个性化 每个医院的模板。将考虑多种统计方法进行匹配，以实现最公平的 比较各医院。Aim 2将使用模拟和真实的患者数据来衡量何时以及为什么 模板匹配和传统的回归方法可能产生对医院的不一致的评估。 目标3将开发模板匹配数据的呈现方式，然后调查VA医学主任以进行比较 使用模板匹配的业绩报告的可解释性和可信度与传统的 回归医院基准，与后续半结构化访谈的一个子集的主任， 从医学主任的角度进一步了解模板匹配性能报告。

Background: Identifying and remediating low-quality care is at the heart of systematic quality improvement, particularly in VA. However, cross-hospital comparisons to identify under-performing hospitals are limited by differences in patient case-mix and illness severity. Clinicians consider the current approach to benchmarking with conventional regression to be unfair, unclear, and unhelpful. While substantial resources are devoted to hospital benchmarking, the current return on this investment is limited because clinicians do not understand or trust the methods. The National Academy of Medicine has recently called for investing into the science of performance measurement and for increasing its transparency and validity. Template matching has been proposed as an alternative methodological approach to benchmarking that is fair, clear, and helpful. However, this new approach has never been tested outside of limited research settings. Specific Aims: To test the utility of template matching for comparing quality of care across VA's diverse acute care hospitals, this project will assess the feasibility, accuracy, and interpretability of this approach. Specifically, the project will: (A1) Feasibility: Develop and optimize two template matching approaches for comparing 30-day mortality across VA hospitals. (A2) Accuracy: Compare the ability of template matching versus conventional regression to correctly identify under-performing hospitals. (A3) Interpretability: Compare the interpretability and credibility of hospital performance data generated from template matching versus conventional regression models with clinical leaders. Anticipated Impact: The overarching goal of this proposal is to improve the care of hospitalized Veterans. This proposal will apply promising, new template matching approaches for benchmarking to the diverse VA healthcare system and complete a multi-faceted evaluation of these approaches. We will consider not only the feasibility and methodological rigor of template matching, but also the interpretability, credibility and accountability of the data. We expect that template matching will be feasible for benchmarking VA acute care hospitals, that it will identify under-performing hospitals at least as well as current benchmarking with conventional regression, and that it will be more interpretable and credible to VA Chiefs of Medicine than current performance reports. We anticipate many future expansions to this work, including the use of template matching for VA to private sector comparisons. Unique Features and Innovation: This proposal is innovative because it will simultaneously (1) advance the statistical state-of-the-art in hospital benchmarking, (2) develop the necessary infrastructure in consultation with operational partners to use template matching in VA, and (3) evaluate this new approach not just statistically, but also with the actual users it is supposed to inform. The results are highly relevant to VA, whose performance is always under scrutiny. It is also of great interest outside VA, for the many healthcare systems who similarly struggle to provide meaningful and actionable assessments of individual hospital's performance. Project Methods: Aim 1 will develop and refine the statistical methodology for benchmarking 30-day mortality in VA acute care hospitals using two template matching approaches: (1) a single template and (2) personalized templates for each hospital. Multiple statistical approaches to matching will be considered to achieve the fairest comparisons across hospitals. Aim 2 will use simulation and real patient data to measure when and why template matching and conventional regression approaches may yield discordant assessments of hospitals. Aim 3 will develop the presentation of template matching data, then survey VA Chiefs of Medicine to compare the interpretability and credibility of performance reports using template matching versus conventional regression for hospital benchmarking, with follow-up semi-structured interviews with a subset of Chiefs to further understand template matching performance reports from the perspective of Chiefs of Medicine.