Medical Image Retrieval: A Multimodal Approach.
Medical Image Retrieval: A Multimodal Approach.
复制标题
医学图像检索：多模态方法
DOI：
10.4137/cin.s14053
复制
发表时间：
2014
影响因子：
2
通讯作者：
Müller H
中科院分区：
文献类型：
--
作者：
Cao Y;Steffey S;He J;Xiao D;Tao C;Chen P;Müller H
医学成像正在成为抗癌战争的重要组成部分。在癌症护理和癌症研究期间，以数字格式捕获和记录了大量的医学图像数据。面对如此海量的异构图像数据，有必要开发有效的基于内容的医学图像检索系统，以满足癌症临床实践和科研的需要。虽然基于内容的图像检索（CBIR）的研究在不同领域取得了实质性的进展，但由于医学图像的独特性，现有的CBIR技术直接应用于医学图像的检索效果并不理想。在本文中，我们开发了一种新的多模态医学图像检索方法的基础上的统计图形模型和深度学习的最新进展。具体来说，我们首先研究了一个新的扩展的概率潜在语义分析模型，整合医学图像的视觉和文本信息，以弥合语义鸿沟。然后，我们开发了一个新的基于深度玻尔兹曼机的多模态学习模型，从多模态信息中学习联合密度模型，以获得缺失的模态。大量真实世界医学图像的实验结果表明，我们的新方法是一个有前途的解决方案，为下一代医学影像索引和检索系统。
Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.