Monitoring and auditing of AI in clinical practice - radiotherapy contour quality assurance

临床实践中人工智能的监测和审核——放疗轮廓质量保证

• 批准号: 10072915
• 金额: $ 6.36万
• 依托单位: INPICTURA LIMITED
• 依托单位国家: 英国
• 项目类别: Grant for R&D
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=10072915
• 关键词: Monitoring; auditing; AI; clinical; practice

Outlining organs on a 3D image, such as a computed tomography (CT) scan, is a necessary step in planning radiotherapy for the treatment of cancer. This is a time consuming and repetitive task. Since 2017, AI-based methods have been introduced to automate this process to a large degree. However, the clinical team must check and edit these outlines, or contours, to ensure they are accurate.In commissioning such tools, clinics perform quantitative evaluation of accuracy or attempt to measure the expected workflow efficiency gain. Existing measures and tools for evaluation of contouring performance rely on global, organ per organ, quantitative contouring similarity measures, comparing the AI-generated contours with those manually delineated by an expert. Time saving is assessed by comparing the current clinical contouring practice to the prospective check-and-edit approach once the AI-solution is introduced into the contouring workflow.After commissioning and clinical implementation, an ongoing quality assurance (QA), i.e. 'post-market surveillance', is needed. This includes the auditing of user interactions in routine clinical use to determine the true performance impact of the AI-model in clinical practice. While AI-based auto-contouring solutions are analysed and evaluated quantitatively in commissioning, no solutions exist to perform ongoing analysis of their performance in clinical practice. Furthermore, monitoring that checking-and-editing is being performed, and not being overlooked, is critical to their safe deployment.In this project, we seek to develop a prototype system for automated, and ongoing, quality assurance and audit of AI-based contouring tools. By using such a tool, clinics will be able to implement AI-based contouring in a safer manner and identify any limitations or biases that may exist in such systems.

在三维图像（如计算机断层扫描（CT）扫描）上勾画器官轮廓，是规划癌症放疗治疗的必要步骤。这是一项耗时且重复的任务。自2017年以来，基于人工智能的方法在很大程度上实现了这一过程的自动化。然而，临床团队必须检查和编辑这些轮廓，或轮廓，以确保它们是准确的。在调试这些工具时，诊所对准确性进行定量评估，或尝试测量预期的工作流程效率增益。评估轮廓性能的现有措施和工具依赖于全局、每个器官、定量轮廓相似性措施，将人工智能生成的轮廓与专家手动描绘的轮廓进行比较。将人工智能解决方案引入轮廓工作流程后，通过比较当前的临床轮廓实践与预期的检查和编辑方法来评估节省的时间。在调试和临床实施后，持续的质量保证（QA），即“上市后监督”是必要的。这包括审计日常临床使用中的用户交互，以确定人工智能模型在临床实践中的真实性能影响。虽然基于人工智能的自动轮廓解决方案在调试中进行了定量分析和评估，但没有解决方案可以在临床实践中对其性能进行持续分析。此外，监视检查和编辑是否被执行，而不是被忽视，对于它们的安全部署至关重要。在这个项目中，我们寻求开发一个原型系统，用于基于人工智能的轮廓工具的自动化和持续的质量保证和审计。通过使用这样的工具，诊所将能够以更安全的方式实施基于人工智能的轮廓，并识别此类系统中可能存在的任何限制或偏见。