Intelligent Monitoring? Assessing the ability of the Care Quality Commission's statistical surveillance tool to predict quality and prioritise NHS hospital inspections
Intelligent Monitoring? Assessing the ability of the Care Quality Commission's statistical surveillance tool to predict quality and prioritise NHS hospital inspections
复制标题
DOI：
10.1136/bmjqs-2015-004687
复制
发表时间：
2017-02-01
影响因子：
5.4
通讯作者：
Rothstein, Henry
中科院分区：
文献类型：
--
作者：
Griffiths, Alex;Beaussier, Anne-Laure;Rothstein, Henry
背景护理质量委员会（CQC）负责确保英格兰30 000多名注册提供者提供的健康和社会护理的质量。由于进行现场检查的资源有限，CQC使用统计监督工具来帮助确定应优先检查哪些供应商。面对计划中的资金削减，CQC计划更多地依赖统计监测工具来评估质量风险，并相应地确定检查优先级。目的评估CQC最新的监测工具智能监测（IM）的能力，预测国家卫生服务（NHS）医院信托提供的护理质量，以便那些提供穷人的风险最大的人-方法通过回归分析和卡方检验评价IM工具的预测能力，并对IM工具产生的定量风险评分与随后由大型专家检查员团队进行详细现场检查后授予的质量评级之间的关系进行卡方检验。由CQC的IM统计监督工具生成的连续风险评分不能预测NHS医院信任的基于检查的质量评级（优秀/良好或0.38（0.14至1.05），或0.94（0.80至-1.10）为良好/需要改善，OR 0.90（0.76至1.07）为需要改善/不充分）。第二，风险分数不能更简单地用来区分信托表现不佳-那些随后被评为“需要改进”或“不足”的信托，以及那些随后被评为“良好”或“优秀”的信托（OR 1.07（0.91至1.26））。将CQC的风险等级1-3分为高风险，4-6分为低风险，11个高风险信托基金表现良好，43个低风险信托基金表现不佳，整体准确率为47.6%。第三，风险评分不能更简单地用于区分表现最差的信托-那些随后被评为“不合格”的信托-与其余的，表现较好的信托（OR 1.11（0.94至1.32））。将CQC的风险等级1归类为高风险，将2-6归类为低风险，实现了最高的总体准确率72.8%，但13个不充分的信托中仍然只有6个被正确归类为高风险。结论由于IM统计监督工具无法预测NHS医院信托检查的结果，因此不能用于优先级排序。因此，需要对视察规划采取新的办法。
Background The Care Quality Commission (CQC) is responsible for ensuring the quality of the health and social care delivered by more than 30 000 registered providers in England. With only limited resources for conducting on-site inspections, the CQC has used statistical surveillance tools to help it identify which providers it should prioritise for inspection. In the face of planned funding cuts, the CQC plans to put more reliance on statistical surveillance tools to assess risks to quality and prioritise inspections accordingly.Objective To evaluate the ability of the CQC's latest surveillance tool, Intelligent Monitoring (IM), to predict the quality of care provided by National Health Service (NHS) hospital trusts so that those at greatest risk of providing poor-quality care can be identified and targeted for inspection.Methods The predictive ability of the IM tool is evaluated through regression analyses and chi(2) testing of the relationship between the quantitative risk score generated by the IM tool and the subsequent quality rating awarded following detailed on-site inspection by large expert teams of inspectors.Results First, the continuous risk scores generated by the CQC's IM statistical surveillance tool cannot predict inspection-based quality ratings of NHS hospital trusts (OR 0.38 (0.14 to 1.05) for Outstanding/Good, OR 0.94 (0.80 to -1.10) for Good/Requires improvement, and OR 0.90 (0.76 to 1.07) for Requires improvement/Inadequate). Second, the risk scores cannot be used more simply to distinguish the trusts performing poorly-those subsequently rated either 'Requires improvement' or 'Inadequate' from the trusts performing well-those subsequently rated either 'Good' or 'Outstanding' (OR 1.07 (0.91 to 1.26)). Classifying CQC's risk bandings 1-3 as high risk and 4-6 as low risk, 11 of the high risk trusts were performing well and 43 of the low risk trusts were performing poorly, resulting in an overall accuracy rate of 47.6%. Third, the risk scores cannot be used even more simply to distinguish the worst performing trusts-those subsequently rated 'Inadequate'-from the remaining, better performing trusts (OR 1.11 (0.94 to 1.32)). Classifying CQC's risk banding 1 as high risk and 2-6 as low risk, the highest overall accuracy rate of 72.8% was achieved, but still only 6 of the 13 Inadequate trusts were correctly classified as being high risk.Conclusions Since the IM statistical surveillance tool cannot predict the outcome of NHS hospital trust inspections, it cannot be used for prioritisation. A new approach to inspection planning is therefore required.