Real-world evaluation of rapid and laboratory-free COVID-19 triage for emergency care: external validation and pilot deployment of artificial intelligence driven screening.
Real-world evaluation of rapid and laboratory-free COVID-19 triage for emergency care: external validation and pilot deployment of artificial intelligence driven screening.
复制标题
DOI：
10.1016/s2589-7500(21)00272-7
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
CURIAL Translational Collaborative
中科院分区：
文献类型：
--
作者：
Soltan AAS;Yang J;Pattanshetty R;Novak A;Yang Y;Rohanian O;Beer S;Soltan MA;Thickett DR;Fairhead R;Zhu T;Eyre DW;Clifton DA;CURIAL Translational Collaborative
患者COVID-19状态的不确定性导致治疗延迟、医院内传播和医院运营压力。然而，实验室PCR的典型周转时间仍然是12-24小时，并且侧流装置（LFD）具有有限的灵敏度。此前，我们已经证明，人工智能驱动的分诊（CURIAL-1.0）可以使用到达医院后1小时内常规可用的临床数据提供快速COVID-19筛查。在这里，我们的目标是改善从到达急诊室到结果可用性的时间，进行外部和前瞻性验证，并在英国急诊室部署一种新的无实验室筛查工具。我们优化了我们以前的模型，删除了信息量较少的预测因子，以提高通用性和速度，开发了具有生命体征和现成血液检查（全血细胞计数[FBC];尿素，肌酐和电解质;肝功能检查;和C反应蛋白）的CURIAL-Lab模型和仅具有生命体征和FBC的CURIAL-Rapide模型。通过与PCR检测进行比较，对伯明翰大学医院、贝德福德郡医院和朴茨茅斯医院大学国民健康服务（NHS）信托基金的急诊入院模型进行了外部验证，并在牛津大学医院进行了前瞻性验证。接下来，我们将模型性能直接与LFD进行比较，并评估了将CURIAL模型结果阳性或LFD阳性的患者分流到COVID-19疑似临床区域的组合路径。最后，我们部署了CURIAL-Rapide和经批准的床旁FBC分析仪，在John Radcliffe医院（英国牛津）提供无实验室COVID-19筛查。我们的主要改善结果是获得结果的时间，我们的性能指标是灵敏度、特异性、阳性和阴性预测值以及受试者工作特征曲线下面积（AUROC）。在2019年12月1日至2021年3月31日的总验证期内，四个验证医院组中有72223名患者符合资格标准。CURIAL-Lab和CURIAL-Rapide在各信托基金中的表现一致（AUROC范围0·858-0·881，95% CI 0·838-0·912，CURIAL-Lab和0·836-0·854，0·814-0·889，CURIAL-Rapide），在朴茨茅斯医院达到最高灵敏度（CURIAL-Lab为84.1%，Wilson 95% CI 82.5 - 85.7，CURIAL-Rapide为83.5%，81.8 - 85.1），CURIAL-Lab和CURIAL-Rapide的特异性分别为71.3%（70.9 - 71.8）和63.6%（63.1 - 64.1）。当与LFD相结合时，模型预测将分类灵敏度从56.9%提高到56.9%。（51·7-62·0），对于单独的LFD，使用CURIAL-Lab时为85·6%（81·6-88·9; AUROC 0·925），CURIAL-Rapide为88·2%（84·4-91·1; AUROC 0·919），从而使用CURIAL-Lab和CURIAL-Rapide分别将COVID-19漏诊病例减少了65%和72%。对于CURIAL-Rapide的前瞻性部署，在2021年2月18日至5月10日期间入组了520例患者进行床旁FBC分析，其中436例接受了确证性PCR检测，10例（2.3%）检测结果呈阳性。从到达到CURIAL-Rapide结果的中位时间为45分钟（IQR 32-64），比LFD（61分钟，37-99;对数秩p<0.0001）快16分钟（26.3%），比PCR（7小时37分钟，6小时5分钟至15小时39分钟; p<0.0001）快6小时52分钟（90.2%）。分类性能较高，敏感性为87.5%（95%CI 52.9 - 97.8），特异性为85.4%（81.3 - 88.7），阴性预测值为99.7%（98.2 - 99.9）。CURIAL-Rapide正确排除了53名患者中的31名（58.5%）感染，这些患者被医生分诊到COVID-19疑似地区，但随后通过PCR检测为阴性。我们的研究结果显示了人工智能驱动的COVID-19筛查在急诊科的通用性、性能和实际运营效益。CURIAL-Rapide在与近患者FBC分析一起使用时提供了快速、免实验室筛查，并能够减少COVID-19检测阴性但被分流到COVID-19疑似地区的患者数量。Wellcome Trust，牛津大学医学和生命科学转化基金。
Uncertainty in patients' COVID-19 status contributes to treatment delays, nosocomial transmission, and operational pressures in hospitals. However, the typical turnaround time for laboratory PCR remains 12–24 h and lateral flow devices (LFDs) have limited sensitivity. Previously, we have shown that artificial intelligence-driven triage (CURIAL-1.0) can provide rapid COVID-19 screening using clinical data routinely available within 1 h of arrival to hospital. Here, we aimed to improve the time from arrival to the emergency department to the availability of a result, do external and prospective validation, and deploy a novel laboratory-free screening tool in a UK emergency department. We optimised our previous model, removing less informative predictors to improve generalisability and speed, developing the CURIAL-Lab model with vital signs and readily available blood tests (full blood count [FBC]; urea, creatinine, and electrolytes; liver function tests; and C-reactive protein) and the CURIAL-Rapide model with vital signs and FBC alone. Models were validated externally for emergency admissions to University Hospitals Birmingham, Bedfordshire Hospitals, and Portsmouth Hospitals University National Health Service (NHS) trusts, and prospectively at Oxford University Hospitals, by comparison with PCR testing. Next, we compared model performance directly against LFDs and evaluated a combined pathway that triaged patients who had either a positive CURIAL model result or a positive LFD to a COVID-19-suspected clinical area. Lastly, we deployed CURIAL-Rapide alongside an approved point-of-care FBC analyser to provide laboratory-free COVID-19 screening at the John Radcliffe Hospital (Oxford, UK). Our primary improvement outcome was time-to-result, and our performance measures were sensitivity, specificity, positive and negative predictive values, and area under receiver operating characteristic curve (AUROC). 72 223 patients met eligibility criteria across the four validating hospital groups, in a total validation period spanning Dec 1, 2019, to March 31, 2021. CURIAL-Lab and CURIAL-Rapide performed consistently across trusts (AUROC range 0·858–0·881, 95% CI 0·838–0·912, for CURIAL-Lab and 0·836–0·854, 0·814–0·889, for CURIAL-Rapide), achieving highest sensitivity at Portsmouth Hospitals (84·1%, Wilson's 95% CI 82·5–85·7, for CURIAL-Lab and 83·5%, 81·8–85·1, for CURIAL-Rapide) at specificities of 71·3% (70·9–71·8) for CURIAL-Lab and 63·6% (63·1–64·1) for CURIAL-Rapide. When combined with LFDs, model predictions improved triage sensitivity from 56·9% (51·7–62·0) for LFDs alone to 85·6% with CURIAL-Lab (81·6–88·9; AUROC 0·925) and 88·2% with CURIAL-Rapide (84·4–91·1; AUROC 0·919), thereby reducing missed COVID-19 cases by 65% with CURIAL-Lab and 72% with CURIAL-Rapide. For the prospective deployment of CURIAL-Rapide, 520 patients were enrolled for point-of-care FBC analysis between Feb 18 and May 10, 2021, of whom 436 received confirmatory PCR testing and ten (2·3%) tested positive. Median time from arrival to a CURIAL-Rapide result was 45 min (IQR 32–64), 16 min (26·3%) sooner than with LFDs (61 min, 37–99; log-rank p<0·0001), and 6 h 52 min (90·2%) sooner than with PCR (7 h 37 min, 6 h 5 min to 15 h 39 min; p<0·0001). Classification performance was high, with sensitivity of 87·5% (95% CI 52·9–97·8), specificity of 85·4% (81·3–88·7), and negative predictive value of 99·7% (98·2–99·9). CURIAL-Rapide correctly excluded infection for 31 (58·5%) of 53 patients who were triaged by a physician to a COVID-19-suspected area but went on to test negative by PCR. Our findings show the generalisability, performance, and real-world operational benefits of artificial intelligence-driven screening for COVID-19 over standard-of-care in emergency departments. CURIAL-Rapide provided rapid, laboratory-free screening when used with near-patient FBC analysis, and was able to reduce the number of patients who tested negative for COVID-19 but were triaged to COVID-19-suspected areas. The Wellcome Trust, University of Oxford Medical and Life Sciences Translational Fund.