Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study.
Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study.
复制标题
DOI：
10.1186/s12913-022-08124-w
复制
发表时间：
2022-06-03
影响因子：
2.8
通讯作者：
Prescott, Hallie C.
中科院分区：
文献类型：
--
作者：
McGrath, Brenda M.;Takamine, Linda;Hogan, Cainnear K.;Hofer, Timothy P.;Rosen, Amy K.;Sussman, Jeremy B.;Wiitala, Wyndy L.;Ryan, Andrew M.;Prescott, Hallie C.
医院特定模板匹配（HS-TM）是一种较新的医院绩效评估方法。评估基于 HS-TM 与基于回归的绩效评估的可解释性、可信度和可用性。我们对医院领导进行了调查（2021 年 1 月至 5 月）并完成了后续半结构化访谈。调查包括四个假设的绩效评估小插曲，方法（HS-TM，回归）和医院死亡率是随机的。全国退伍军人事务部参谋长、医学部和医院医学部主任。正确的解释；自评对解释的信心；以及对评估的自评信任度（通过调查）。对可信度和主要用途的担忧（通过采访记录的主题分析）。总共 84 名参与者完成了 295 个调查小插曲。受访者正确解释 HS-TM 的比例为 81.8%，回归评估的比例为 56.5%，p < 0.001。受访者“相信结果”的 HS-TM 比例为 70.9%，回归评估比例为 58.2%，p = 0.03。确定了关于可信度的九个担忧：对病例组合和/或疾病严重程度的捕捉不足；无法解释专门项目（例如移植中心）；与地理位置不同的医院进行比较；将死亡率与质量等同起来；缺乏评判标准；低功率；与不同医院的比较；排名的生成；以及缺乏透明度。有五个问题与这两种方法同样相关，其中一个与 HS-TM 更相关，另外三个与回归更相关。评估主要用于引发进一步的质量评估（“检查油灯”）并激励行为改变。与基于回归的评估相比，基于 HS-TM 的绩效评估对于 VA 医院领导来说更容易解释、更可信。然而，领导者对这两种方法的可信度有一系列类似的担忧，并认为最好将这两种方法用作进一步评估的筛选。在线版本包含可在 10.1186/s12913-022-08124-w 获取的补充材料。
Hospital-specific template matching (HS-TM) is a newer method of hospital performance assessment. To assess the interpretability, credibility, and usability of HS-TM-based vs. regression-based performance assessments. We surveyed hospital leaders (January-May 2021) and completed follow-up semi-structured interviews. Surveys included four hypothetical performance assessment vignettes, with method (HS-TM, regression) and hospital mortality randomized. Nationwide Veterans Affairs Chiefs of Staff, Medicine, and Hospital Medicine. Correct interpretation; self-rated confidence in interpretation; and self-rated trust in assessment (via survey). Concerns about credibility and main uses (via thematic analysis of interview transcripts). In total, 84 participants completed 295 survey vignettes. Respondents correctly interpreted 81.8% HS-TM vs. 56.5% regression assessments, p < 0.001. Respondents “trusted the results” for 70.9% HS-TM vs. 58.2% regression assessments, p = 0.03. Nine concerns about credibility were identified: inadequate capture of case-mix and/or illness severity; inability to account for specialized programs (e.g., transplant center); comparison to geographically disparate hospitals; equating mortality with quality; lack of criterion standards; low power; comparison to dissimilar hospitals; generation of rankings; and lack of transparency. Five concerns were equally relevant to both methods, one more pertinent to HS-TM, and three more pertinent to regression. Assessments were mainly used to trigger further quality evaluation (a “check oil light”) and motivate behavior change. HS-TM-based performance assessments were more interpretable and more credible to VA hospital leaders than regression-based assessments. However, leaders had a similar set of concerns related to credibility for both methods and felt both were best used as a screen for further evaluation. The online version contains supplementary material available at 10.1186/s12913-022-08124-w.
登录
查看更多内容
影响因子：
3.2
作者：
Hsieh, HF;Shannon, SE
通讯作者：
Shannon, SE
影响因子：
3
作者：
Vincent BM;Molling D;Escobar GJ;Hofer TP;Iwashyna TJ;Liu VX;Rosen AK;Ryan AM;Seelye S;Wiitala WL;Prescott HC
通讯作者：
Prescott HC
影响因子：
3.4
作者：
Silber, Jeffrey H.;Rosenbaum, Paul R.;Fleisher, Lee A.
通讯作者：
Fleisher, Lee A.
影响因子：
5.7
作者：
Prescott, Hallie C.;Kadel, Rajendra P.;Eyman, Julie R.;Freyberg, Ron;Quarrick, Matthew;Brewer, David;Hasselbeck, Rachael
通讯作者：
Hasselbeck, Rachael
影响因子：
9.7
作者：
Fihn, Stephan D.;Francis, Joseph;Graham, Gail L.
通讯作者：
Graham, Gail L.