Monitoring and auditing of AI in clinical practice - radiotherapy contour quality assurance

点击翻译按钮获取中文摘要

Outlining organs on a 3D image, such as a computed tomography (CT) scan, is a necessary step in planning radiotherapy for the treatment of cancer. This is a time consuming and repetitive task. Since 2017, AI-based methods have been introduced to automate this process to a large degree. However, the clinical team must check and edit these outlines, or contours, to ensure they are accurate.In commissioning such tools, clinics perform quantitative evaluation of accuracy or attempt to measure the expected workflow efficiency gain. Existing measures and tools for evaluation of contouring performance rely on global, organ per organ, quantitative contouring similarity measures, comparing the AI-generated contours with those manually delineated by an expert. Time saving is assessed by comparing the current clinical contouring practice to the prospective check-and-edit approach once the AI-solution is introduced into the contouring workflow.After commissioning and clinical implementation, an ongoing quality assurance (QA), i.e. 'post-market surveillance', is needed. This includes the auditing of user interactions in routine clinical use to determine the true performance impact of the AI-model in clinical practice. While AI-based auto-contouring solutions are analysed and evaluated quantitatively in commissioning, no solutions exist to perform ongoing analysis of their performance in clinical practice. Furthermore, monitoring that checking-and-editing is being performed, and not being overlooked, is critical to their safe deployment.In this project, we seek to develop a prototype system for automated, and ongoing, quality assurance and audit of AI-based contouring tools. By using such a tool, clinics will be able to implement AI-based contouring in a safer manner and identify any limitations or biases that may exist in such systems.