A unifying approach to machine learning driven medical image segmentation

医学图像包含大量关于患者解剖结构和疾病的信息。根据预期用途：诊断或干预，可将其大致分为两类。医学图像分割，或轮廓，在医学研究和临床护理中起着关键作用，但非常耗时和昂贵。受过训练的专家使用专门的软件来手动描绘图像结构的边界，例如，在计算机断层扫描（CT）图像中的心脏。单个CT图像是256个以上图像切片的体积，每个切片中可能有10个或更多结构要分割。在研究中，分割是量化图像或一系列图像中结构随时间推移的形状和外观的重要第一步。然而，诊断成像不是常规轮廓的临床护理，从而大大限制了其研究潜力，没有昂贵的轮廓。例如，要回答一个看似简单的问题，COVID-19是否会导致感染前和感染后患者的心脏或肺部形状发生变化，在1，000名患者的数据集中可能需要超过256，000个轮廓和700小时的专业劳动（假设每个切片10秒），这对数万名患者的大型研究造成了重大障碍。为了充分发挥加拿大庞大的诊断医学图像库的研究潜力，需要自动分割技术。分割也可以在临床护理中发挥关键作用。在诊断图像中，自动分割可用于更好地量化病变区域。相比之下，许多医疗程序（例如，癌症患者的放射治疗）需要分割以进行干预计划。在这些临床工作流程中，自动分割可以提高效率，在不增加成本的情况下实现更大的吞吐量，通过减少等待时间和降低医疗保健系统成本使加拿大人受益。迄今为止，由于技术限制，医学图像分割方法仅限于在由几百名患者组成的专用研究数据集上进行训练，对可能的结构数量有很大限制，对于给定模型通常为9个或更少。我们建议通过增加其可推广性来推进最先进的医学图像分割方法，以从数千个常规临床放射治疗图像和超过77个不同结构中构建模型。这项研究不仅将推进医学图像分割方法的知识，还将释放加拿大数据集的更大潜力，并开发更强大的分割模型，使我们更接近于在研究和临床工作流程中使用的分割解决方案。

Medical images contain a plethora of information about patient anatomy and disease. They can be roughly broken into two categories based on intended use: diagnostic, or intervention. Medical image segmentation, or contouring, plays a key role in both medical research and clinical care but is very time consuming and costly. A trained expert uses specialized software to manually delineate the boundary of a structure an image, e.g., the heart in a computed tomography (CT) image. A single CT image is a volume of upwards of 256 image slices with perhaps 10 or more structures to segment in each. In research, segmentation is a vital first step to quantify the shape, and appearance of a structure in an image or series of images over time. However, diagnostic imaging is not routinely contoured for clinical care, thus greatly limiting its research potential without expensive contouring. For example, to answer a seemingly simple question, does COVID-19 cause a change in the shape of the heart or lungs in patients pre-and post- infection across a dataset of 1,000 patients could require upwards of 256,000 contours and 700 hours of expert labor (assuming 10 seconds per slice) creating a significant barrier to larger studies with tens of thousands of patients. To unlock the full research potential of Canada's vast repositories of diagnostic medical images automated segmentation technologies are required. Segmentation can also play a key role in clinical care. In diagnostic images automated segmentation could be used to better quantify diseased areas. In contrast, many medical procedures (for example radiation therapy for cancer patients) require segmentation for intervention planning. In these clinical workflows automated segmentation could improve efficiency enabling greater throughput without a cost increase, benefiting Canadians through reduced wait times and reduced healthcare system costs. To date due to technical limitations medical image segmentation methods have been limited to training on purpose-built research datasets consisting of a few hundred patients, with a heavy limit on the number of possible structures, typically 9 or fewer for a given model. We propose to advance state-of-the-art methodologies for medical image segmentation by increasing their generalizability to building models from thousands of routine clinical radiation therapy images, and more than 77 distinct structures. This research will not only advance knowledge in medical image segmentation methodologies but unlock the greater potential of datasets in Canada and develop more robust segmentation models brining us closer to a solved solution for segmentation to use in both research and clinical workflows.