PET and MRI image fusion based on a dense convolutional network with dual attention
PET and MRI image fusion based on a dense convolutional network with dual attention
复制标题
基于双重关注的密集卷积网络的 PET 和 MRI 图像融合
DOI：
10.1016/j.compbiomed.2022.106339
复制
发表时间：
2022-11-29
影响因子：
7.7
通讯作者：
Wang, Zongmin
中科院分区：
文献类型：
--
作者：
Li, Bicao;Hwang, Jenq-Neng;Wang, Zongmin
医学图像中不同模态的融合技术，例如正电子发射断层扫描（PET）和磁共振成像（MRI），通过集成不同医学图像的互补信息，在许多临床应用中变得越来越重要。在本文中，我们提出了一种基于双重关注密集卷积网络（CSpA-DN）的新型 PET 和 MRI 图像融合模型。在我们的框架中，构建了由密集连接的神经网络组成的编码器来从源图像中提取特征，并采用解码器网络从这些特征中生成融合图像。同时，在编码器和解码器中引入了双重注意模块，以进一步自适应地集成局部特征及其全局依赖性。在双注意力模块中，利用空间注意力块通过所有位置的特征信息的加权和从编码器网络中提取每个点的特征。同时，所有图像特征的相互依赖的相关性通过通道注意力模块进行聚合。此外，我们设计了特定的损失函数，包括图像损失、结构损失、梯度损失和感知损失，以保留更多的结构和细节信息并锐化目标的边缘。我们的方法有助于融合图像不仅保留 PET 图像的丰富功能信息，而且保留 MRI 图像的丰富细节结构。根据定性观察和客观评估，公开数据集上的实验结果说明了 CSpA-DN 模型与最先进方法相比的优越性。
The fusion techniques of different modalities in medical images, e.g., Positron Emission Tomography (PET) and Magnetic Resonance Imaging (MRI), are increasingly significant in many clinical applications by integrating the complementary information from different medical images. In this paper, we propose a novel fusion model based on a dense convolutional network with dual attention (CSpA-DN) for PET and MRI images. In our framework, an encoder composed of the densely connected neural network is constructed to extract features from source images, and a decoder network is employed to generate the fused image from these features. Simultaneously, a dual-attention module is introduced in the encoder and decoder to further integrate local features along with their global dependencies adaptively. In the dual-attention module, a spatial attention block is leveraged to extract features of each point from encoder network by a weighted sum of feature information at all positions. Meanwhile, the interdependent correlation of all image features is aggregated via a module of channel attention. In addition, we design a specific loss function including image loss, structural loss, gradient loss and perception loss to preserve more structural and detail information and sharpen the edges of targets. Our approach facilitates the fused images to not only preserve abundant functional information from PET images but also retain rich detail structures of MRI images. Experimental results on publicly available datasets illustrate the superiorities of CSpA-DN model compared with state-of-the-art methods according to both qualitative observation and objective assessment.