Accelerating adoption of trustworthy AI in radiology: scalable software for non-technical clinical users to independently validate commercial products at local sites

我们提出了一个全新的解决方案，允许医疗保健提供者和医院以独立的方式快速评估和测试放射人工智能产品，而资源成本只有当前评估框架的一小部分。不断增长和老龄化的人口，以及更长期的健康状况，正在给医疗保健系统带来越来越大的压力。如果安全有效地采用人工智能，可以缓解这种情况。然而，目前检验商业人工智能产品可信度和性能的途径既不标准化，也不具有可扩展性。超过80%的医院途径涉及影像及其放射科医生的专家解释。然而，仅英格兰就缺少1,453名临床放射科医生(2021年)，临床放射科主任报告说，安全有效的患者护理人数不足。[11].临床放射学中的人工智能和机器学习工具有望提高发现率，简化临床工作流程，并提高患者的安全性。然而，尽管早期的热情很高，但应用到常规实践中的速度一直很慢\[2\]。一个关键原因是，在部署之前，每个产品都必须经过外部和独立的本地数据验证，以确保可重复性、通用性和可信性，以及操作性能和故障点。然而，许多评价放射学人工智能产品的研究和试验受到偏见、缺乏盲目性以及数据集和人口数量较小的阻碍。此外，人工智能产品在所谓的“黑匣子”中运行，外行用户对产品的操作方法不清楚，这就造成了可信度和可信性方面的问题。这给寻求部署人工智能的机构带来了巨大的成本和潜在的临床风险，为广泛采用制造了一大障碍。显然，开发资源密集度较低的流程的需求尚未得到满足，组织可以通过这些流程来证实医疗设备制造商的声明，并确定为医疗保健提供者提供明显临床益处的产品。我们建议创建一个标准化系统，在采购之前在当地临床站点验证商业医疗成像AI产品。我们的解决方案评估了黑盒产品的性能、公平性、健壮性和可解释性。非技术临床经理可以使用这一点来快速评估和比较人工智能产品，确保适合当地人口和工作方案。

We propose a novel and radical solution that allows healthcare providers and hospitals to rapidly evaluate and test radiology AI products in an independent manner, at a fraction of the resource cost of current evaluation frameworks.An increasing and aging population, with more long-term health conditions, is putting increasing strain on the healthcare system. AI can alleviate this if adopted safely and effectively \[1\]. However, current pathways to examine the trustworthiness and performance of commercial AI products are neither standardised nor scalable.Over 80% of hospital pathways involve imaging and its expert interpretation by radiologists \[11\]. However, England alone had a shortfall of 1,453 clinical radiologist consultants (2021), with clinical radiology directors reporting insufficient numbers for safe and effective patient care. \[11\]. AI and machine learning tools in clinical radiology promise to improve detection rates, streamline clinical workflow and improve patient safety. However, despite early enthusiasm, adoption into routine practice has been slow \[2\].A key reason for this is that, prior to deployment, each product must be externally and independently validated on local data to ensure reproducibility, generalisability and trustworthiness, along with operational performance and points of failure \[3\]. However, many studies and trials evaluating radiology AI products are hampered by bias, lack of blinding and small datasets and population numbers \[5\]. Furthermore, AI products operate in so-called "black-boxes" whereby the method in which the product operates is unclear to the lay user, creating issues around credibility and trustworthiness.This poses significant cost and potential clinical risk to institutions looking to deploy AI, creating a major barrier to widespread adoption. There is a clear unmet need to develop less resource intensive processes by which organizations can substantiate medical device manufacturer claims and identify products that offer clear clinical benefit to healthcare providers.We propose to create a standardised system to validate commercial healthcare imaging AI products at local clinical sites before procurement. Our solution assesses the performance, fairness, robustness and explainability of a black-box product. This can be used by non-technical clinical managers to quickly evaluate and compare AI products, ensuring suitability for their local populations and work schemes.