Multimodal medical image fusion based on discrete Tchebichef moments and pulse coupled neural network

Multimodal medical image fusion plays a vital role in clinical diagnoses and treatment planning. In many image fusion methods‐based pulse coupled neural network (PCNN), normalized coefficients are used to motivate the PCNN, and this makes the fused image blur, detail loss, and decreases contrast. Moreover, they are limited in dealing with medical images with different modalities. In this article, we present a new multimodal medical image fusion method based on discrete Tchebichef moments and pulse coupled neural network to overcome the aforementioned problems. First, medical images are divided into equal‐size blocks and the Tchebichef moments are calculated to characterize image shape, and energy of blocks is computed as the sum of squared non‐DC moment values. Then to retain edges and textures, the energy of Tchebichef moments for blocks is introduced to motivate the PCNN with adaptive linking strength. Finally, large firing times are selected as coefficients of the fused image. Experimental results show that the proposed scheme outperforms state‐of‐the‐art methods and it is more effective in processing medical images with different modalities. © 2017 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 27, 57–65, 2017     

Improving medical image fusion method using fuzzy entropy and nonsubsampling contourlet transform

Many types of medical images must be fused, as single‐modality medical images can only provide limited information due to the imaging principles and the complexity of human organ structures. In this paper, a multimodal medical image fusion method that combines the advantages of nonsubsampling contourlet transform (NSCT) and fuzzy entropy is proposed to provide a basis for clinical diagnosis and improve the accuracy of target recognition and the quality of fused images. An image is initially decomposed into low‐ and high‐frequency subbands through NSCT. The corresponding fusion rules are adopted in accordance with the different characteristics of the low‐ and high‐frequency components. The membership degree of low‐frequency coefficients is calculated. The fuzzy entropy is also computed and subsequently used to guide the fusion of coefficients to preserve image details. High‐frequency components are fused by maximizing the regional energy. The final fused image is obtained by inverse transformation. Experimental results show that the proposed method achieves good fusion effect based on the subjective visual effect and objective evaluation criteria. This method can also obtain high average gradient, SD, and edge preservation and effectively retain the details of the fused image. The results of the proposed algorithm can provide effective reference for doctors to assess patient condition.     

基于NSST和SWT的红外与可见光图像融合算法研究

目的鉴于非下采样剪切波变换NSST的红外与可见光图像融合的结果存在细微特征缺失问题,提出一种基于NSST和SWT的红外与可见光图像融合算法,以提升融合图像的质量。方法首先分别对红外与可见光图像进行NSST分解,各得到一个低频系数和多个不同方向、尺度的高频系数。然后低频系数分别通过SWT分解得到新的低频系数和高频系数,通过SWT分解得到的新的低频系数和高频系数分别采用采用线性加权平均法和区域平均能量取大的融合策略,融合结果再进行SWT逆变换得到低频系数融合结果。高频系数采用区域平均能量取大的融合策略进行融合。最后通过NSST逆变换得到最终的融合图像。结果通过仿真实验结果表明,文中算法与NSST,SWT和NSCT等算法相比,融合图像在主观视觉上的红外目标更突出,图像细节更清晰,且在IE, AG, QAB/F, SF和SD等评价指标上也最优。结论文中算法的融合结果能更好地表现源图像的目标信息和细节纹理信息,表明该算法具有优越性。     

Multimodal sensor medical image fusion based on mutual‐structure for joint filtering using sparse representation

Multimodal sensor medical image fusion has been widely reported in recent years, but the fused image by the existing methods introduces low contrast information and little detail information. To overcome this problem, the new image fusion method is proposed based on mutual‐structure for joint filtering and sparse representation in this article. First, the source image is decomposed into a series of detail images and coarse images by mutual‐structure for joint filtering. Second, sparse representation is adopted to fuse coarse images and then local contrast is applied for fusing detail images. Finally, the fused image is reconstructed by the addition of the fused coarse images and the fused detail images. By experimental results, the proposed method shows the best performance on preserving detail information and contrast information in the views of subjective and objective evaluations.     

Statistical correlative model in the multimodal fusion of brain images

Fusing multimodal medical images into an integrated image, providing more details and rich information thereby facilitating medical diagnosis and therapy. Most of the existing multiscale-based fusion methods ignore the correlations between the decomposition coefficients and lead to incomplete fusion results. A novel contextual hidden Markov model (CHMM) is proposed to construct the statistical model of contourlet coefficients. First, the pair brain images are decomposed into multiscale, multidirectional, and anisotropic subbands with a contourlet transform. Then the low-frequency components are fused with the choose-max rule. For the high-frequency coefficients, the CHMM is learned with the EM algorithm, and incorporate with a novel fuzzy entropy-based context, building the fuzzy relationships among these coefficients. Finally, the fused brain image is obtained by using the inverse contourlet transform. Fusion experiments on several multimodal brain images show the superiority of the proposed method in terms of both visual quality and some widely used objective measures.     

Image fusion methodology for efficient interpretation of multiband images in 3D high‐resolution ultrasonic transmission tomography

With the critical innovations of using submillimeter transducers and multiband analysis of the first arrival pulse, a high‐resolution ultrasonic transmission tomography (HUTT) system has been built and tested to produce multiband images of biological organs at submillimeter resolution. Since the resulting multiband images consist of frequency‐dependent attenuation coefficients (relative to water reference) of transmitted ultrasound pulses, their contrast and sharpness depend on the specific frequency band(s) used for image formation. Even though this multiband representation provides a powerful tool for soft‐tissue differentiation, it hinders visual inspection and limits the visual interpretation of image contents in a short time. To facilitate the visual interpretation of HUTT multiband images, this article presents an efficient image fusion methodology called local principal component analysis with structure tensor (LPCA‐ST). The LPCA has been known as a feasible tool for the fusion of spectral data, since it utilizes the principal components of spectral data as a fusion‐weighting vector of local area. Nonetheless, the LPCA‐fused image often suffers from oversmoothness because of the redundancy of the spectral data. To prevent this problem, we propose a structure tensor as the metric used to select the most informative bands for subsequent LPCA fusion. Our preliminary studies have shown that the contrast of the LPCA‐fused image improves dramatically only when multiband images whose values of the respective structure tensor are the highest are used in the LPCA fusion process. This is achieved in 3D without increasing the computational complexity of the fusion process. © 2009 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 19, 277–282, 2009     

A novel multimodal medical image fusion using sparse representation and modified spatial frequency

Fusion of multimodal imaging data supports medical experts with ample information for better disease diagnosis and further clinical investigations. Recently, sparse representation (SR)‐based fusion algorithms has been gaining importance for their high performance. Building a compact, discriminative dictionary with reduced computational effort is a major challenge to these algorithms. Addressing this key issue, we propose an adaptive dictionary learning approach for fusion of multimodal medical images. The proposed approach consists of three steps. First, zero informative patches of source images are discarded by variance computation. Second, the structural information of remaining image patches is evaluated using modified spatial frequency (MSF). Finally, a selection rule is employed to separate the useful informative patches of source images for dictionary learning. At the fusion step, batch‐OMP algorithm is utilized to estimate the sparse coefficients. A novel fusion rule which measures the activity level in both spatial domain and transform domain is adopted to reconstruct the fused image with the sparse vectors and trained dictionary. Experimental results of various medical image pairs and clinical data sets reveal that the proposed fusion algorithm gives better visual quality and competes with existing methodologies both visually and quantitatively.     

Medical image fusion based on nuclear norm minimization

Medical image fusion plays an important role in diagnosis and treatment of diseases such as image‐guided radiotherapy and surgery. Although numerous medical image fusion methods have been proposed, most approaches have not touched the low rank nature of matrix formed by medical image, which usually lead to fusion image distortion and image information loss. These methods also often lack universality when dealing with different kinds of medical images. In this article, we propose a novel medical image fusion to overcome aforementioned issues on existing methods with the aid of low rank matrix approximation with nuclear norm minimization (NNM) constraint. The workflow of our method is described as: firstly, nonlocal similar patches across the medical image are searched by block matching for local patch in source images. Second, a fused matrix is stacking by shared nonlocal similarity patches, then the low rank matrix approximation methods under nuclear norm minimization can be used to recover low rank feature of fused matrix. Finally, fused image can be gotten by aggregating all the fused patches. Experimental results show that the proposed method is superior to other methods in both subjectively visual performance and objective criteria. © 2015 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 25, 310–316, 2015     

基于PCNN的图像融合新方法

本文提出了一种基于PCNN的新型图像融合算法.首先,对待融合的两幅图像进行平稳小波分解得到两组多尺度图像;接着,取其中任意一组作为主PCNN的输入、另一组相应的图像作为从PCNN的输入,在每次迭代时,经并行PCNN点火后,得到一系列多尺度融合图像;然后,对它们进行平稳小波反变换得到每次迭代的融合结果;最后,计算每次迭代结果的信息熵,取信息熵值最大的融合图像作为最终结果.大量的实验以及与其它融合算法的比较分析,表明了本文算法的有效性和优越性.     

Medical image fusion based on improved sum‐modified‐Laplacian

Sum‐modified‐Laplacian (SML) plays an important role in medical image fusion. However, fused rules based on larger SML always lead to fusion image distortion in transform domain image fusion or image information loss in spatial domain image fusion. Combined with average filter and median filter, a new medical image fusion method based on improved SML (ISML) is proposed. First, a basic fused image is gained by ISML, which is used for evaluation of the selection map of medical images. Second, difference images can be obtained by subtracting average image of all sources of medical images. Finally, basic fused image can be refined by difference images. The algorithm can both preserve the information of the source images well and suppress pixel distortion. Experimental results demonstrate that the proposed method outperforms the state‐of‐the‐art medical image fusion methods. © 2015 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 25, 206–212, 2015     

基于边缘信息的偏振图像融合算法及评价

偏振遥感图像通常都采用强度、偏振度、偏振角来表征目标偏振特性.本文提出的基于边缘信息的偏振图像融合算法是将三幅偏振图像利用离散小波变换把图像分解成不同尺度的低频和高频部分,采用小波区域窗口和子区域窗口统计把小波系数分类成边缘和非边缘系数,通过这些方法进行有效的边缘细节信息提取.在融合处理中,低频图像的小波系数平均值作为融合后的低频系数,高频细节系数根据不同区域特征选择方法以及对应输入图像小波系数的窗口区域方差来确定融合后高频小波系数.仿真实验结果表明,这样使得融合后的图像细节更真实更丰富,图像的偏振特性体现更为充分,同时减少对源图像的预处理要求,使图像在整体上有较好的视觉效果.从而证明这种方法能够在保留图像微小细节方面获得满意的结果,且算法有效性优于其他的图像融合方法.     

Optimal multi-scale geometric fusion based on non-subsampled contourlet transform and modified central force optimization

In the current era of technological development, medical imaging plays an important part in several applications of medical diagnosis and therapy. This requires more precise images with much more details and information for correct medical diagnosis and therapy. Medical image fusion is one of the solutions for obtaining much spatial and spectral information in a single image. This article presents an optimization-based contourlet image fusion approach in addition to a comparative study for the performance of both multi-resolution and multi-scale geometric effects on fusion quality. An optimized multi-scale fusion technique based on the Non-Subsampled Contourlet Transform (NSCT) using the Modified Central Force Optimization (MCFO) and local contrast enhancement techniques is presented. The first step in the proposed fusion approach is the histogram matching of one of the images to the other to allow the same dynamic range for both images. The NSCT is used after that to decompose the images to be fused into their coefficients. The MCFO technique is used to determine the optimum decomposition level and the optimum gain parameters for the best fusion of coefficients based on certain constraints. Finally, an additional contrast enhancement process is applied on the fused image to enhance its visual quality and reinforce details. The proposed fusion framework is subjectively and objectively evaluated with different fusion quality metrics including average gradient, local contrast, standard deviation (STD), edge intensity, entropy, peak signal-to-noise ratio, Q ^ab/f, and processing time. Experimental results demonstrate that the proposed optimized NSCT medical image fusion approach based on the MCFO and histogram matching achieves a superior performance with higher image quality, average gradient, edge intensity, STD, better local contrast and entropy, a good quality factor, and much more details in images. These characteristics help for more accurate medical diagnosis in different medical applications.     

Intelligent Fusion of Infrared and Visible Image Data Based on Convolutional Sparse Representation and Improved Pulse-Coupled Neural Network

Multi-source information can be obtained through the fusion of infrared images and visible light images, which have the characteristics of complementary information. However, the existing acquisition methods of fusion images have disadvantages such as blurred edges, low contrast, and loss of details. Based on convolution sparse representation and improved pulse-coupled neural network this paper proposes an image fusion algorithm that decompose the source images into high-frequency and low-frequency subbands by non-subsampled Shearlet Transform (NSST). Furthermore, the low-frequency subbands were fused by convolutional sparse representation (CSR), and the high-frequency subbands were fused by an improved pulse coupled neural network (IPCNN) algorithm, which can effectively solve the problem of difficulty in setting parameters of the traditional PCNN algorithm, improving the performance of sparse representation with details injection. The result reveals that the proposed method in this paper has more advantages than the existing mainstream fusion algorithms in terms of visual effects and objective indicators.     

Fusion of MRI and CT images using guided image filter and image statistics

In medical imaging using different modalities such as MRI and CT, complementary information of a targeted organ will be captured. All the necessary information from these two modalities has to be integrated into a single image for better diagnosis and treatment of a patient. Image fusion is a process of combining useful or complementary information from multiple images into a single image. In this article, we present a new weighted average fusion algorithm to fuse MRI and CT images of a brain based on guided image filter and the image statistics. The proposed algorithm is as follows: detail layers are extracted from each source image by using guided image filter. Weights corresponding to each source image are calculated from the detail layers with help of image statistics. Then a weighted average fusion strategy is implemented to integrate source image information into a single image. Fusion performance is assessed both qualitatively and quantitatively. Proposed method is compared with the traditional and recent image fusion methods. Results showed that our algorithm yields superior performance.     

基于二代Curvelet变换耦合二维因子的图像融合算法

目的 为了解决当前图像融合算法在融合过程中忽略了低频系数中所包含的图像细节信息,导致其输出的融合图像存在间断以及模糊效应的不足,方法 提出基于二代Curvelet变换耦合二维因子的图像融合算法。首先,利用具有多尺度以及多方向特性的二代Curvelet变换对源图像进行快速的分解,以获取源图像精细的低频以及高频系数。引入低频系数的信息熵以及区域方差特征来构造二维因子,对低频系数所包含的信息量以及图像的变化程度进行度量,以完成低频系数的融合。随后,利用高频系数的平均梯度特征,构造信息融合规则,完成高频系数的融合,提高融合图像的细节信息含有量。最后,利用像素点的R,G,B值,构造颜色校正因子,对融合图像进行颜色修正,以获取色彩效果较好的融合图像。结果 实验结果显示,与当前图像融合算法相比,所提算法具有更强的细节表达能力,其输出的融合图像具有更好的清晰度及视觉效果。结论 所提算法拥有较好的融合质量,能提高图像的对比度与分辨率,在图像处理领域具有一定的参考价值。     

Multi-focus image fusion based on the non-subsampled contourlet transform

In this paper, a new image fusion algorithm based on non-subsampled contourlet transform (NSCT) is proposed for the fusion of multi-focus images. The selection of different subband coefficients obtained by the NSCT decomposition is critical to image fusion. So, in this paper, firstly, original images are decomposed into different frequency subband coefficients by NSCT. Secondly, the selection of the low-frequency subband coefficients and the bandpass directional subband coefficients is discussed in detail. For the selection of the low-frequency subband coefficients, the non-negative matrix factorization (NMF) method is adopted. For the selection of bandpass directional subband coefficients, a regional cross-gradient method that selects the coefficients according to the minimum of the regional cross-gradient is proposed. Finally, the fused image is obtained by performing the inverse NSCT on the combined coefficients. The experimental results show that the proposed fusion algorithm can achieve significant results in getting a new image where all parts are sharp.     

基于多目标粒子群算法的多传感器图像融合

针对图像融合中参数优化的问题,提出了一种基于多目标粒子群优化算法的多传感器图像融合方法。首先采用非采样Contourlet变换(NSCT)对源图像进行多尺度、多方向分解;然后选取图像融合的客观评价指标为优化目标函数,采用多目标粒子群优化算法对低频系数的融合参数进行优化,带通方向子带系数采用取绝对值最大的融合规则;最后通过NSCT逆变换得到融合图像。分别对多聚焦图像融合和红外与可见光图像进行融合实验,并对融合图像进行主客观评价,实验结果表明,得到的融合图像具有较好的主观视觉效果和客观评价指标。     

一种NSCT域多聚焦图像融合新方法

针对多聚焦图像融合存在的问题,提出一种基于非下采样Contourlet变换（NSCT）的多聚焦图像融合新方法。首先,采用NSCT对多聚焦图像进行分解;然后,对低频系数采用基于改进拉普拉斯能量和（SML）的视觉特征对比度进行融合,对高频系数采用基于二维Log-Gabor能量进行融合;最后,对得到的融合系数进行重构得到融合图像。实验结果表明,无论是运用视觉的主观评价,还是基于互信息、边缘信息保留值等客观评价标准,该文所提方法都优于传统的离散小波变换、平移不变离散小波变换、NSCT等融合方法。     

基于方向滤波器组与Laplacian能量和的图像融合算法

目的针对基于Contourlet变换的融合算法在边缘上易出现吉布斯现象,使其融合图像产生几何失真的问题,设计一种非下采样方向滤波器组耦合局部Laplacian能量和的图像融合算法。方法首先,结合多小波变换(multi-wavelet transform,MWT)与非下采样方向滤波器组(Non-Subsampled Direction FilterBank,NSDFB),将图像分解为3个高频方向系数和1个低频系数。对于低频系数,采用局部修正的Laplacian能量和(Local Sum-Modified-Laplacian,LSML)与脉冲耦合神经网络(Pulse couple neural network,PCNN)组合的LSML-PCNN模型来完成低频信息的融合。对于高频系数,通过提取低频和高频子带边缘,并利用系数绝对最大值法作为依据,实现高频系数的融合。结果实验数据表明,与当前图像融合方案相比,所提算法具有更高的融合质量,得到的融合图像边缘更加清晰和完整。结论所提算法拥有较高的融合视觉效果,可改善图像的对比度和分辨率,在图像处理领域具有一定的参考价值。     

Modified Global Flower Pollination Algorithm‐based image fusion for medical diagnosis using computed tomography and magnetic resonance imaging

Recently, the computed tomography (CT) and magnetic resonance imaging (MRI) medical image fusion have turned into a challenging issue in the medical field. The optimal fused image is a significant component to detect the disease easily. In this research, we propose an iterative optimization approach for CT and MRI image fusion. Initially, the CT and MRI image fusion is subjected to a multilabel optimization problem. The main aim is to minimize the data and smoothness cost during image fusion. To optimize the fusion parameters, the Modified Global Flower Pollination Algorithm is proposed. Here, six sets of fusion images with different experimental analysis are evaluated in terms of different evaluation metrics such as accuracy, specificity, sensitivity, SD, structural similarity index, feature similarity index, mutual information, fusion quality, and root mean square error (RMSE). While comparing to state‐of‐art methods, the proposed fusion model provides best RMSE with higher fusion performance. Experiments on a set of MRI and CT images of medical data set show that the proposed method outperforms a very competitive performance in terms of fusion quality.