Automatic labelling of anatomies in large-scale medical image datasets through self-supervised and multimodal learning

近年来，基于新型深度学习方法的医疗卷数据分析取得了很大进展。然而，要想在医疗保健应用中成功和广泛地部署，反映大量人口的横截面并可靠地识别正常解剖和异常的大规模数据集是缺失的。该项目的科学目标是基于大规模NAKO人口研究的30,000卷3D MRI扫描，开发强大，自动和高效的内部器官，骨骼和体表分割算法。在这里，基于学习的多模态注册方法和从先前DFG项目的无注释3D数据集学习的方法将在软件演示中进一步开发和利用。在这种知识转移的背景下，项目合作伙伴将系统地扩展Fraunhofer MEVIS和l<s:1>贝克大学的互补先前工作，以新颖的深度学习方法整合自监督预训练，多模式迁移学习，图像配准和分割的组合。然后，这些将应用于从NAKO人口研究中自动分割数千个3D MRI体积，并与手动金标准相比，在验证数据集中对它们进行评估。与应用合作伙伴飞利浦一起，将从分割模型中估计不确定性和异常情况，并创建3D几何图谱，该图谱将使用点云网络来定位基于身体表面的内部解剖结构。在UzL开发的神经网络的源代码和训练模型，以及NAKO数据的解剖标签将免费提供给研究界。在项目期间，将共同创建一个演示器（TRL 6-7），它将实现临床常规MRI采集过程的更高自动化，使用基于深度图像的基于表面的解剖识别，提供显着的经济优势，并加速采集过程，并通过扫描的自动分析实现进一步的开发可能性。

The analysis of medical volume data has made great progress in recent years based on novel deep learning methods. However, large-scale datasets that reflect a large cross-section of the population and reliably recognise both normal anatomy and abnormalities are missing for successful and wide-spread deployment in healthcare applications. The scientific goal of the project is to develop robust, automatic and efficient algorithms for the segmentation of internal organs, bones and body surfaces based on 3D MRI scans from the large-scale NAKO population study with 30,000 volumes. Here, methods of learning-based multimodal registration and learning from non-annotated 3D datasets from the preceding DFG project will be further developed and exploited in a software demonstrator. In the context of this knowledge transfer, the project partners will systematically extend the complementary prior work of Fraunhofer MEVIS and the University of Lübeck to integrate a combination of self-supervised pre-training, multimodal transfer learning, image registration and segmentation in novel deep learning methods. These will then be applied to automatically segment several thousand 3D MRI volumes from the NAKO population study and evaluate them on a validation dataset compared to the manual gold standard. Together with the application partner Philips, uncertainties and anomalies will be estimated from the segmentation models and a 3D geometric atlas will be created that will use point cloud networks to localise internal anatomies based on body surfaces. Source code and trained models, of the neural networks developed at UzL, as well as anatomical labels of the NAKO data will be made freely available to the research community. Together, a demonstrator (TRL 6-7) will be created during the project, which will realise a higher automation of the MRI acquisition process in clinical routine using surface-based anatomy recognition based on depth images, offering a significant economic advantage and as well as an acceleration of the acquisition process and enabling further exploitation possibilities through the automatic analysis of the scans.