Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restoration.
Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restoration.
复制标题
DOI：
10.1007/978-3-030-59710-8_14
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Liang, Jianming
中科院分区：
文献类型：
--
作者：
Haghighi, Fatemeh;Hosseinzadeh Taher, Mohammad Reza;Zhou, Zongwei;Gotway, Michael B;Liang, Jianming
医学图像自然与人体解剖学的丰富语义相关联，反映在大量重复出现的解剖模式中，为促进深度语义表示学习和为不同的医学应用产生语义上更强大的模型提供了独特的潜力。但是，如何将医学图像中嵌入的这种强大而自由的语义用于自我监督学习，在很大程度上尚未探索。为此，我们训练深度模型，通过自我发现、自我分类和自我恢复医学图像下的解剖结构来学习语义丰富的视觉表示，从而生成语义丰富的通用预训练3D模型，称为Semantic Genesis。我们使用所有公开的预训练模型，通过自我监督或完全监督，在六个不同的目标任务上检查我们的语义起源，涵盖各种医疗模式中的分类和分割（即，CT、MRI和X射线）。我们广泛的实验表明，Semantic Genesis显著超过了所有3D同行，以及事实上的基于ImageNet的2D迁移学习。这种表现归功于我们新颖的自监督学习框架，鼓励深度模型从医学图像中嵌入的一致解剖结构产生的丰富解剖模式中学习引人注目的语义表示。代码和预先训练的语义起源可在https://github.com/JLiangLab/SemanticGenesis上获得。
Medical images are naturally associated with rich semantics about the human anatomy, reflected in an abundance of recurring anatomical patterns, offering unique potential to foster deep semantic representation learning and yield semantically more powerful models for different medical applications. But how exactly such strong yet free semantics embedded in medical images can be harnessed for self-supervised learning remains largely unexplored. To this end, we train deep models to learn semantically enriched visual representation by self-discovery, self-classification, and self-restoration of the anatomy underneath medical images, resulting in a semantics-enriched, general-purpose, pre-trained 3D model, named Semantic Genesis. We examine our Semantic Genesis with all the publicly-available pre-trained models, by either self-supervision or fully supervision, on the six distinct target tasks, covering both classification and segmentation in various medical modalities (i.e., CT, MRI, and X-ray). Our extensive experiments demonstrate that Semantic Genesis significantly exceeds all of its 3D counterparts as well as the de facto ImageNet-based transfer learning in 2D. This performance is attributed to our novel self-supervised learning framework, encouraging deep models to learn compelling semantic representation from abundant anatomical patterns resulting from consistent anatomies embedded in medical images. Code and pre-trained Semantic Genesis are available at https://github.com/JLiangLab/SemanticGenesis.