Identifying quality improvement intervention evaluations: is consensus achievable?
Identifying quality improvement intervention evaluations: is consensus achievable?
复制标题
DOI：
10.1136/qshc.2009.036475
复制
发表时间：
2010-08-01
影响因子：
--
通讯作者：
Shekelle, P. G.
中科院分区：
文献类型：
--
作者：
Danz, M. S.;Rubenstein, L. V.;Shekelle, P. G.
背景质量改进干预措施（QIIs）的多样性阻碍了使用证据审查来推进质量改进活动。商定的确定QII文章的框架将促进证据审查和围绕最佳做法达成共识。目的适应和测试证据审查方法，以确定适用于评估QII有效性、影响或成功的经验性QII评估。设计文献检索，测量多层次评价者之间的一致性和评价不一致。方法对2005-2007年出版的10篇期刊进行电子检索，并按标题和摘要进行筛选。然后，三对审稿人从筛选列表中随机选择22篇文章，进行独立评分。评估Kappa统计数据和百分比一致性。质量改进的12个利益相关者，包括QII专家和期刊编辑，对审稿人不同意的出版物进行评级和讨论。结果评价者对QII开发、实施或结果的实证评价的一致性为73% （kappa值低至0.041）。评分者和利益攸关方就如何改进协议进行的讨论侧重于三个有争议的文章选择问题：没有关于患者健康、提供者行为或护理结果过程的数据；没有证据表明干预措施适应当地情况；一个设计只使用观察方法，作为相关分析，没有对照组。结论审稿人的认同程度仅为中等。对相关文章的可靠鉴定是评估已发表证据的第一步。质量改进的进展将取决于基于理论和共识的开发和测试用于识别QII评估的可推广框架。
Background The diversity of quality improvement interventions (QIIs) has impeded the use of evidence review to advance quality improvement activities. An agreed-upon framework for identifying QII articles would facilitate evidence review and consensus around best practices.Aim To adapt and test evidence review methods for identifying empirical QII evaluations that would be suitable for assessing QII effectiveness, impact or success.Design Literature search with measurement of multilevel inter-rater agreement and review of disagreement.Methods Ten journals (2005-2007) were searched electronically and the output was screened based on title and abstract. Three pairs of reviewers then independently rated 22 articles, randomly selected from the screened list. Kappa statistics and percentage agreement were assessed. 12 stakeholders in quality improvement, including QII experts and journal editors, rated and discussed publications about which reviewers disagreed.Results The level of agreement among reviewers for identifying empirical evaluations of QII development, implementation or results was 73% (with a paradoxically low kappa of 0.041). Discussion by raters and stakeholders regarding how to improve agreement focused on three controversial article selection issues: no data on patient health, provider behaviour or process of care outcomes; no evidence for adaptation of an intervention to a local context; and a design using only observational methods, as correlational analyses, with no comparison group.Conclusion The level of reviewer agreement was only moderate. Reliable identification of relevant articles is an initial step in assessing published evidence. Advancement in quality improvement will depend on the theory-and consensus-based development and testing of a generalizable framework for identifying QII evaluations.