Measuring the variability of AI and manual delineations for radiotherapy

1.对研究背景的简要说明，包括潜在的影响：放射治疗是癌症治疗的一个重要组成部分，但其有效性往往受到不准确的解剖描述所产生的不确定性的影响。这些不准确可能会对肿瘤控制和正常组织毒性的风险产生重大影响。传统上，临床医生手动进行这些描绘；然而，基于人工智能的自动化描绘软件的出现提供了一个有希望的替代方案。这些人工智能解决方案有可能通过节省时间和减少划定的可变性来提高效率。因此，迫切需要制定方法来评估和比较人工和人工智能生成的划定。目的和目标：这项研究的核心目标是开发尖端的机器学习方法，用于量化和表征不同患者群体中描述的可变性。这些方法将能够评估人工和人工智能生成的划定。具体目标如下：-创建最先进的机器学习模型，可以评估解剖轮廓的可变性并将其参数化。-促进不同患者队列的人工和人工生成的轮廓之间的公平和客观比较。-对个别患者的轮廓进行批判性评估，并确定它们是否在更广泛人群中观察到的变异范围内。-简化临床试验和常规临床实践中轮廓的质量保证过程，从而提高治疗效果和患者的安全性。研究方法的新颖性：准确评估描述的可变性是一项艰巨的任务。从历史上看，对轮廓可变性的研究通常遵循相同的方法：对不同的人进行有限数量的扫描，任务是在他们身上描绘相同的结构。一旦划定，就采用指标来量化等高线之间的差异，便于测量观察者之间的变异性(而观察者内部的变异性可以通过让同一人多次描绘扫描来评估)。然而，这种方法受到几个限制。在许多扫描中使用它是不切实际的，参与者意识到自己是研究的一部分，这可能会导致有意识或潜意识地努力表现得比他们在临床环境中更好。此外，这种方法不适合评估人工智能系统，因为人们希望人工智能对同一扫描始终产生相同的结果。因此，该项目的主要创新之一是我们打算设计方法，在每次扫描只经历一次描绘时量化可变性。我们的方法建立在机器学习和深度学习领域的最新进展的基础上，特别是关注不确定性建模和标准化建模。与EPSRC的战略和研究领域保持一致：我们的研究通过推动数据科学和人工智能在医疗保健领域的整合，与EPSRC的战略重点密切一致。具体地说，我们的工作解决了EPSRC对变革性医疗技术、数据驱动的研究以及为改善患者结果而开发的创新方法的重视。5.参与的公司或合作者：-伦敦大学学院医学成像计算中心(CMIC)-国家物理实验室(NPL)-放射治疗试验质量保证小组(RTTQA)-芒特弗农癌症中心(MVCC)

1. Brief description of the context of the research including potential impact:Radiotherapy is a crucial component of cancer treatment, but its effectiveness is often compromised by uncertainties stemming from inaccurate anatomical delineations. These inaccuracies can have significant implications for both tumour control and the risk of normal tissue toxicity. Traditionally, clinicians performed these delineations manually; however, the emergence of AI-based software for automated delineation offers a promising alternative. These AI solutions have the potential to enhance efficiency by saving time and reducing variability in delineations. Therefore, there is a pressing need to devise methods for the assessment and comparison of manual and AI-generated delineations.2. Aims and objectives:The core objective of this research is to develop cutting-edge machine learning methods for quantifying and characterising variability in delineations across a diverse patient population. These methods will be capable of evaluating both manual and AI-generated delineations. The specific aims are as follows:- To create state-of-the-art machine learning models that can assess and parameterise variability in anatomical delineations.- To facilitate fair and objective comparisons between manual and AI-generated delineations for various patient cohorts.- To critically evaluate delineations for individual patients and determine if they fall within the range of variability observed in the broader population.- To streamline the quality assurance process for delineations in clinical trials and routine clinical practice, thereby enhancing treatment efficacy and patient safety.3. Novelty of research methodology:Precisely assessing the variability in delineations is a difficult task. Historically, studies on contour variability usually followed the same method: a limited number of scans were used with various individuals tasked to delineate the same structures on them. Once delineated, metrics were employed to quantify differences between the contours, facilitating the measurement of inter-observer variability (while intra-observer variability could be assessed by having the same individual delineate a scan multiple times). However, this approach suffers from several limitations. It is impractical for use on many scans, and participants' awareness of being part of a study may lead to conscious or subconscious efforts to perform better than they would in a clinical setting. Furthermore, this methodology is unsuitable for evaluating AI systems, as one would expect AI to consistently produce identical results for the same scan.One of the primary innovations in this project is therefore our intention to devise methodologies for quantifying variability when each scan undergoes delineation only once. Our approach builds upon recent advancements in the domains of Machine Learning and Deep Learning, particularly focusing on uncertainty modelling and normative modelling.4. Alignment to EPSRC's strategies and research areas:Our research aligns closely with the strategic priorities of the EPSRC by advancing the integration of data science and artificial intelligence in healthcare. Specifically, our work addresses the EPSRC's emphasis on transformative healthcare technologies, data-driven research, and the development of innovative methodologies for improving patient outcomes. 5. Companies or collaborators involved:- Centre for Medical Imaging Computing (CMIC) at UCL- National Physical Laboratory (NPL)- Radiotherapy Trials Quality Assurance Group (RTTQA)- Mount Vernon Cancer Centre (MVCC)"