Automatic labelling of anatomies in large-scale medical image datasets through self-supervised and multimodal learning

通过自我监督和多模态学习自动标记大规模医学图像数据集中的解剖结构

• 批准号: 500498869
• 负责人: Professor Dr. Mattias Heinrich
• 金额: --
• 依托单位: Fraunhofer-Institut für Digitale Medizin (MEVIS)
• 依托单位国家: 德国
• 项目类别: Research Grants (Transfer Project)
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/500498869?language=en
• 关键词: Automatic; labelling; anatomies; large; scale

The analysis of medical volume data has made great progress in recent years based on novel deep learning methods. However, large-scale datasets that reflect a large cross-section of the population and reliably recognise both normal anatomy and abnormalities are missing for successful and wide-spread deployment in healthcare applications. The scientific goal of the project is to develop robust, automatic and efficient algorithms for the segmentation of internal organs, bones and body surfaces based on 3D MRI scans from the large-scale NAKO population study with 30,000 volumes. Here, methods of learning-based multimodal registration and learning from non-annotated 3D datasets from the preceding DFG project will be further developed and exploited in a software demonstrator. In the context of this knowledge transfer, the project partners will systematically extend the complementary prior work of Fraunhofer MEVIS and the University of Lübeck to integrate a combination of self-supervised pre-training, multimodal transfer learning, image registration and segmentation in novel deep learning methods. These will then be applied to automatically segment several thousand 3D MRI volumes from the NAKO population study and evaluate them on a validation dataset compared to the manual gold standard. Together with the application partner Philips, uncertainties and anomalies will be estimated from the segmentation models and a 3D geometric atlas will be created that will use point cloud networks to localise internal anatomies based on body surfaces. Source code and trained models, of the neural networks developed at UzL, as well as anatomical labels of the NAKO data will be made freely available to the research community. Together, a demonstrator (TRL 6-7) will be created during the project, which will realise a higher automation of the MRI acquisition process in clinical routine using surface-based anatomy recognition based on depth images, offering a significant economic advantage and as well as an acceleration of the acquisition process and enabling further exploitation possibilities through the automatic analysis of the scans.

近年来，基于新颖的深度学习方法，医疗体数据的分析取得了长足的进步。然而，在医疗保健应用中成功和广泛部署时，缺少反映大范围人口并可靠识别正常解剖结构和异常的大规模数据集。该项目的科学目标是开发强大、自动和高效的算法，根据 30,000 卷大规模 NAKO 人群研究的 3D MRI 扫描来分割内脏、骨骼和身体表面。在这里，基于学习的多模态注册方法以及从先前 DFG 项目的无注释 3D 数据集中学习的方法将在软件演示器中得到进一步开发和利用。在这种知识转移的背景下，项目合作伙伴将系统地扩展弗劳恩霍夫 MEVIS 和吕贝克大学之前的互补工作，将自监督预训练、多模态迁移学习、图像配准和分割的组合整合到新颖的深度学习方法中。然后，这些将用于自动分割 NAKO 人群研究中的数千个 3D MRI 体积，并与手动黄金标准相比，在验证数据集上对其进行评估。将与应用合作伙伴飞利浦一起，根据分割模型估计不确定性和异常情况，并创建 3D 几何图集，该图集将使用点云网络根据身体表面定位内部解剖结构。 UzL 开发的神经网络的源代码和训练模型以及 NAKO 数据的解剖标签将免费提供给研究界。项目期间将共同创建一个演示器 (TRL 6-7)，它将使用基于深度图像的表面解剖识别在临床常规中实现 MRI 采集过程的更高自动化，从而提供显着的经济优势，并加速采集过程，并通过扫描的自动分析实现进一步开发的可能性。