What is the suitability of clinical vignettes in benchmarking the performance of online symptom checkers? An audit study.
What is the suitability of clinical vignettes in benchmarking the performance of online symptom checkers? An audit study.
复制标题
DOI：
10.1136/bmjopen-2021-053566
复制
发表时间：
2022-04-27
期刊：
影响因子：
2.9
通讯作者：
Majeed, Azeem
中科院分区：
文献类型：
--
作者：
El-Osta, Austen;Webber, Iman;Alaa, Aos;Bagkeris, Emmanouil;Mian, Saba;Sharabiani, Mansour Taghavi Azar;Majeed, Azeem
评估临床小插曲对在线症状检查器（OSCs）的性能进行基准测试的适用性。使用公开的免费OSC进行观察性研究。健康OSC，提供英语咨询，被用来记录咨询结果，从两个外行和四个专家输入使用139个标准化的病人小样本。每个小插图包括三种诊断解决方案和分类建议在三类分类紧急。一个由三名独立的全科医生组成的小组解释了这些小插曲，以得出一套替代的诊断和分诊解决方案。这两组诊断和分类解决方案进行了整合，形成了用于基准测试的最终整合版本。六个输入器使用健康OSC模拟了834个标准化的患者评估，并记录了输出（分诊方案、路标，以及正确的诊断方案是首先出现还是在前三个差异中出现）。我们估计Cohen的kappa来评估不同输入者的解释如何导致不同的OSC输出，即使使用相同的小插曲或与单独的医生小组进行比较。在分诊建议上有中等程度的一致性（kappa=0.48），在所有输入者的咨询结果上有实质性的一致性（kappa=0.73）。与最终的综合诊断和分诊方案相比，OSC的性能从基线显著提高（p<0.001）。临床小插曲本身是有限的，他们的效用基准诊断准确性或分诊安全OSC。涉及真实患者的真实世界证据研究被推荐用于对照一组医生对OSC的性能进行基准测试。
Assess the suitability of clinical vignettes in benchmarking the performance of online symptom checkers (OSCs). Observational study using a publicly available free OSC. Healthily OSC, which provided consultations in English, was used to record consultation outcomes from two lay and four expert inputters using 139 standardised patient vignettes. Each vignette included three diagnostic solutions and a triage recommendation in one of three categories of triage urgency. A panel of three independent general practitioners interpreted the vignettes to arrive at an alternative set of diagnostic and triage solutions. Both sets of diagnostic and triage solutions were consolidated to arrive at a final consolidated version for benchmarking. Six inputters simulated 834 standardised patient evaluations using Healthily OSC and recorded outputs (triage solution, signposting, and whether the correct diagnostic solution appeared first or within the first three differentials). We estimated Cohen’s kappa to assess how interpretations by different inputters could lead to divergent OSC output even when using the same vignette or when compared with a separate panel of physicians. There was moderate agreement on triage recommendation (kappa=0.48), and substantial agreement on consultation outcomes between all inputters (kappa=0.73). OSC performance improved significantly from baseline when compared against the final consolidated diagnostic and triage solution (p<0.001). Clinical vignettes are inherently limited in their utility to benchmark the diagnostic accuracy or triage safety of OSC. Real-world evidence studies involving real patients are recommended to benchmark the performance of OSC against a panel of physicians.
登录
查看更多内容
影响因子：
4.7
作者：
North, Frederick;Ward, William J.;Tulledge-Scheitel, Sidna M.
通讯作者：
Tulledge-Scheitel, Sidna M.
影响因子：
5.5
作者：
Sheppard, J.;Kumar, K.;Raza, K.
通讯作者：
Raza, K.
获得可持续和适合的英国健康和护理人员。
影响因子：
168.9
作者：
Anderson, Michael;O'Neill, Ciaran;Clark, Jill Macleod;Street, Andrew;Woods, Michael;Johnston-Webber, Charlotte;Charlesworth, Anita;Whyte, Moira;Foster, Margaret;Majeed, Azeem;Pitchforth, Emma;Mossialos, Elias;Asaria, Miqdad;McGuire, Alistair
通讯作者：
McGuire, Alistair
数字症状评估应用程序在建议条件和紧急建议方面的准确性如何？临床小插图与GPS进行了比较。
影响因子：
2.9
作者：
Gilbert S;Mehl A;Baluch A;Cawley C;Challiner J;Fraser H;Millen E;Montazeri M;Multmeier J;Pick F;Richter C;Türk E;Upadhyay S;Virani V;Vona N;Wicks P;Novorol C
通讯作者：
Novorol C
DOI：
10.1016/j.ijchp.2014.12.001
发表时间：
2015-05-01
影响因子：
8.8
作者：
Evans, Spencer C.;Roberts, Michael C.;Reed, Geoffrey M.
通讯作者：
Reed, Geoffrey M.