Formative evaluation of the accuracy of a clinical decision support system for cervical cancer screening.
Formative evaluation of the accuracy of a clinical decision support system for cervical cancer screening.
复制标题
DOI：
10.1136/amiajnl-2013-001613
复制
发表时间：
2013-07
期刊：
影响因子：
--
通讯作者：
Chaudhry R
中科院分区：
文献类型：
--
作者：
Wagholikar KB;MacLaughlin KL;Kastner TM;Casey PM;Henry M;Greenes RA;Liu H;Chaudhry R
我们之前开发并报道了一个用于宫颈癌筛查的原型临床决策支持系统(CDSS)。然而，该系统是复杂的，因为它基于多准则和自由文本处理。因此，该系统容易出现故障。本报告描述了对该系统的形成性评估，这是确保该系统部署就绪的必要步骤。作为CDSS的潜在最终用户的护理提供者被邀请为代表不同决策情景的随机患者集提供他们的建议。将护理提供者的建议与CDSS产生的建议进行了比较。两名独立专家审查了不匹配的建议。共有25名用户参与了这项研究，并为175例患者提供了建议。CDSS的准确率为87%，并确定了12种CDSS错误，这主要是由于该系统的指南规则存在缺陷。当缺陷被纠正后，CDSS为所有失败案例生成了最佳建议，但文件不完整的案例除外。建立参考集的众包办法，加上对不匹配建议的专家审查，通过查明系统开发人员遗漏的决策情景，促进了对该系统的有效评价和改进。所描述的方法将对其他寻求快速评估和增强复杂决策支持系统的部署准备情况的研究人员有用。
We previously developed and reported on a prototype clinical decision support system (CDSS) for cervical cancer screening. However, the system is complex as it is based on multiple guidelines and free-text processing. Therefore, the system is susceptible to failures. This report describes a formative evaluation of the system, which is a necessary step to ensure deployment readiness of the system. Care providers who are potential end-users of the CDSS were invited to provide their recommendations for a random set of patients that represented diverse decision scenarios. The recommendations of the care providers and those generated by the CDSS were compared. Mismatched recommendations were reviewed by two independent experts. A total of 25 users participated in this study and provided recommendations for 175 cases. The CDSS had an accuracy of 87% and 12 types of CDSS errors were identified, which were mainly due to deficiencies in the system's guideline rules. When the deficiencies were rectified, the CDSS generated optimal recommendations for all failure cases, except one with incomplete documentation. The crowd-sourcing approach for construction of the reference set, coupled with the expert review of mismatched recommendations, facilitated an effective evaluation and enhancement of the system, by identifying decision scenarios that were missed by the system's developers. The described methodology will be useful for other researchers who seek rapidly to evaluate and enhance the deployment readiness of complex decision support systems.
登录
查看更多内容
DOI：
10.1136/amiajnl-2010-000063
发表时间：
2011-05-01
影响因子：
6.4
作者：
Hoeksema, Laura J.;Bazzy-Asaad, Alia;Horwitzi, Leora I.
通讯作者：
Horwitzi, Leora I.
影响因子：
7.2
作者：
Lee, Jennifer Wai-Yin;Berkowitz, Zahava;Saraiya, Mona
通讯作者：
Saraiya, Mona
DOI：
10.1136/jamia.1996.96342648
发表时间：
1996-01-01
影响因子：
6.4
作者：
Lee, F;Teich, JM;Bates, DW
通讯作者：
Bates, DW
影响因子：
7.2
作者：
Berkowitz, Zahava;Saraiya, Mona;Yabroff, K. Robin
通讯作者：
Yabroff, K. Robin
DOI：
10.1111/j.1467-842x.2005.tb00765.x
发表时间：
2005-06-01
影响因子：
3.5
作者：
Corkrey, R;Parkinson, L;Htun, AT
通讯作者：
Htun, AT