A general framework for image fusion based on multi-scale transform and sparse representation
A general framework for image fusion based on multi-scale transform and sparse representation
复制标题
基于多尺度变换和稀疏表示的图像融合通用框架
DOI：
10.1016/j.inffus.2014.09.004
复制
发表时间：
2015-07-01
影响因子：
18.6
通讯作者：
Wang, Zengfu
中科院分区：
文献类型：
--
作者：
Liu, Yu;Liu, Shuping;Wang, Zengfu
在图像融合文献中，多尺度变换(MST)和稀疏表示(SR)是两种应用最广泛的信号/图像表示理论。为了同时克服基于MST和SR的图像融合方法的固有缺陷，提出了一种将MST和SR相结合的通用图像融合框架。在我们的融合框架中，首先对每一幅预先配准的源图像进行MST，以获得它们的低通和高通系数。然后，使用基于随机共振的融合方法对低通频段进行融合，而对高通频段使用系数绝对值作为活跃度度量进行融合。最后通过对合并系数进行逆MST得到融合图像。首先从理论上详细展示了该融合框架相对于单个基于MST或SR的融合方法的优势，然后通过多聚焦、可见光-红外和医学图像融合的实验验证了该融合框架的优越性。特别是对拉普拉斯金字塔(LP)、低通金字塔比(RP)、离散小波变换(DWT)、双树复小波变换(DTCWT)、曲波变换(CVT)和非下采样轮廓波变换(NSCT)等6种常用的多尺度变换进行了实验测试。通过对融合结果的主客观比较，给出了在所提出的融合框架下，每类图像融合效果最好的融合方法。此外，还研究了滑动窗口步长的影响。实验结果表明，该融合框架能够获得最好的融合效果，尤其适用于多模式图像的融合。(C)2014爱思唯尔B.V.保留所有权利。
In image fusion literature, multi-scale transform (MST) and sparse representation (SR) are two most widely used signal/image representation theories. This paper presents a general image fusion framework by combining MST and SR to simultaneously overcome the inherent defects of both the MST- and SR-based fusion methods. In our fusion framework, the MST is firstly performed on each of the pre-registered source images to obtain their low-pass and high-pass coefficients. Then, the low-pass bands are merged with a SR-based fusion approach while the high-pass bands are fused using the absolute values of coefficients as activity level measurement. The fused image is finally obtained by performing the inverse MST on the merged coefficients. The advantages of the proposed fusion framework over individual MST- or SR-based method are first exhibited in detail from a theoretical point of view, and then experimentally verified with multi-focus, visible-infrared and medical image fusion. In particular, six popular multi-scale transforms, which are Laplacian pyramid (LP), ratio of low-pass pyramid (RP), discrete wavelet transform (DWT), dual-tree complex wavelet transform (DTCWT), curvelet transform (CVT) and nonsubsampled contourlet transform (NSCT), with different decomposition levels ranging from one to four are tested in our experiments. By comparing the fused results subjectively and objectively, we give the best-performed fusion method under the proposed framework for each category of image fusion. The effect of the sliding window's step length is also investigated. Furthermore, experimental results demonstrate that the proposed fusion framework can obtain state-of-the-art performance, especially for the fusion of multimodal images. (C) 2014 Elsevier B.V. All rights reserved.