Improving medical image fusion method using fuzzy entropy and nonsubsampling contourlet transform

Many types of medical images must be fused, as single‐modality medical images can only provide limited information due to the imaging principles and the complexity of human organ structures. In this paper, a multimodal medical image fusion method that combines the advantages of nonsubsampling contourlet transform (NSCT) and fuzzy entropy is proposed to provide a basis for clinical diagnosis and improve the accuracy of target recognition and the quality of fused images. An image is initially decomposed into low‐ and high‐frequency subbands through NSCT. The corresponding fusion rules are adopted in accordance with the different characteristics of the low‐ and high‐frequency components. The membership degree of low‐frequency coefficients is calculated. The fuzzy entropy is also computed and subsequently used to guide the fusion of coefficients to preserve image details. High‐frequency components are fused by maximizing the regional energy. The final fused image is obtained by inverse transformation. Experimental results show that the proposed method achieves good fusion effect based on the subjective visual effect and objective evaluation criteria. This method can also obtain high average gradient, SD, and edge preservation and effectively retain the details of the fused image. The results of the proposed algorithm can provide effective reference for doctors to assess patient condition.     

一种NSCT域多聚焦图像融合新方法

针对多聚焦图像融合存在的问题,提出一种基于非下采样Contourlet变换（NSCT）的多聚焦图像融合新方法。首先,采用NSCT对多聚焦图像进行分解;然后,对低频系数采用基于改进拉普拉斯能量和（SML）的视觉特征对比度进行融合,对高频系数采用基于二维Log-Gabor能量进行融合;最后,对得到的融合系数进行重构得到融合图像。实验结果表明,无论是运用视觉的主观评价,还是基于互信息、边缘信息保留值等客观评价标准,该文所提方法都优于传统的离散小波变换、平移不变离散小波变换、NSCT等融合方法。     

A novel multi-modal medical image fusion method based on shift-invariant shearlet transform

Abstract

Medical image fusion plays an important role in clinical applications, such as image-guided surgery, image-guided radiotherapy, non-invasive diagnosis and treatment planning. Shearlet is a novel multi-scale geometric analysis (MGA) tool proposed recently. In order to overcome the drawback of the shearlet-based fusion methods that the pseudo-Gibbs phenomenon is easily caused around the singularities of the fused image, a new multi-modal medical image fusion method is proposed in shift-invariant shearlet transform domain. First, the original images are decomposed into lowpass sub-bands and highpass sub-bands; then, the lowpass sub-bands and high sub-bands are combined according to the fusion rules, respectively. All the operations are performed in shift-invariant shearlet domain. The final fused image is obtained by directly applying inverse shift-invariant shearlet transform to the fused lowpass sub-bands and highpass sub-bands. Experimental results demonstrate that the proposed method can not only suppress the pseudo-Gibbs phenomenon efficiently, but perform better than the popular wavelet transform-based method, contourlet transform-based method and non-subsampled contourlet transform-based method.     

Multifocus noisy image fusion using contourlet transform

Abstract

Image fusion is a challenging area of research with a variety of applications. The process of image fusion collects information from different sources and combines them in a single composite image. The composite fused image can better describe the scene than any of the source images. In this paper, we have proposed a method for noisy image fusion in contourlet domain. The proposed method works equally well for fusion of noise free images. Contourlet transform is a multiscale, multidirectional transform with various aspect ratios. These properties make it more suitable for image fusion than other conventional transforms. In the proposed work, the fusion algorithm is combined with a denoising algorithm to reverse the effect of noise. In the proposed method, we have used a level dependent threshold that is based on standard deviation of contourlet coefficients, mean and median of the absolute contourlet coefficients. Experimental results demonstrate that the proposed method performs well in the presence of different types of noise. Performance of the proposed method is compared with principal components analysis and sharp fusion based methods as well as other fusion methods based on variants of wavelet transform like dual tree complex wavelet transform, discrete wavelet transform, lifting wavelet transform, multiwavelet transform, stationary wavelet transform and pyramid transform using six standard quantitative quality metrics (entropy, standard deviation, edge strength, fusion factor, sharpness and peak signal to noise ratio). The combined qualitative and quantitative evaluation of the experimental results shows that the proposed method performs better than other methods.     

IGM‐based perceptual multimodal medical image fusion using free energy motivated adaptive PCNN

Multimodal medical image fusion merges two medical images to produce a visual enhanced fused image, to provide more accurate comprehensive pathological information to doctors for better diagnosis and treatment. In this article, we present a perceptual multimodal medical image fusion method with free energy (FE) motivated adaptive pulse coupled neural network (PCNN) by employing Internal Generative Mechanism (IGM). First, source images are divided into predicted layers and detail layers with Bayesian prediction model. Then to retain human visual system inspired features, FE is used to motivate the PCNN for processing detail layers, and large firing times are selected as coefficients. The predicted layers are fused with the averaging strategy as activity level measurement. Finally, the fused image is reconstructed by merging coefficients in both fused layers. Experimental results visually and quantitatively show that the proposed fusion strategy is superior to the state‐of‐the‐art methods.     

Multi-focus image fusion based on the non-subsampled contourlet transform

In this paper, a new image fusion algorithm based on non-subsampled contourlet transform (NSCT) is proposed for the fusion of multi-focus images. The selection of different subband coefficients obtained by the NSCT decomposition is critical to image fusion. So, in this paper, firstly, original images are decomposed into different frequency subband coefficients by NSCT. Secondly, the selection of the low-frequency subband coefficients and the bandpass directional subband coefficients is discussed in detail. For the selection of the low-frequency subband coefficients, the non-negative matrix factorization (NMF) method is adopted. For the selection of bandpass directional subband coefficients, a regional cross-gradient method that selects the coefficients according to the minimum of the regional cross-gradient is proposed. Finally, the fused image is obtained by performing the inverse NSCT on the combined coefficients. The experimental results show that the proposed fusion algorithm can achieve significant results in getting a new image where all parts are sharp.     

A novel multimodal medical image fusion using sparse representation and modified spatial frequency

Fusion of multimodal imaging data supports medical experts with ample information for better disease diagnosis and further clinical investigations. Recently, sparse representation (SR)‐based fusion algorithms has been gaining importance for their high performance. Building a compact, discriminative dictionary with reduced computational effort is a major challenge to these algorithms. Addressing this key issue, we propose an adaptive dictionary learning approach for fusion of multimodal medical images. The proposed approach consists of three steps. First, zero informative patches of source images are discarded by variance computation. Second, the structural information of remaining image patches is evaluated using modified spatial frequency (MSF). Finally, a selection rule is employed to separate the useful informative patches of source images for dictionary learning. At the fusion step, batch‐OMP algorithm is utilized to estimate the sparse coefficients. A novel fusion rule which measures the activity level in both spatial domain and transform domain is adopted to reconstruct the fused image with the sparse vectors and trained dictionary. Experimental results of various medical image pairs and clinical data sets reveal that the proposed fusion algorithm gives better visual quality and competes with existing methodologies both visually and quantitatively.     

基于向量小波与神经网络的图像融合算法

提出了一种基于向量小波和神经网络的图像融合算法.首先对各源图像进行向量小波变换,根据变换后系数计算出各子块图像的清晰度,选取子块图像部分区域清晰度作为前溃神经网络的训练样本,调整神经网络权重;然后用训练好的神经网络组合融合图像的向量小波系数,对组合后的系数进行一致性校验;最后对该系数进行向量小波逆变换,得到融合图像.仿真实验表明,该算法能够较好地解决多传感器图像融合问题,生成的融合图像效果优于有代表性的图像融合方法.     

Optimal multi-scale geometric fusion based on non-subsampled contourlet transform and modified central force optimization

In the current era of technological development, medical imaging plays an important part in several applications of medical diagnosis and therapy. This requires more precise images with much more details and information for correct medical diagnosis and therapy. Medical image fusion is one of the solutions for obtaining much spatial and spectral information in a single image. This article presents an optimization-based contourlet image fusion approach in addition to a comparative study for the performance of both multi-resolution and multi-scale geometric effects on fusion quality. An optimized multi-scale fusion technique based on the Non-Subsampled Contourlet Transform (NSCT) using the Modified Central Force Optimization (MCFO) and local contrast enhancement techniques is presented. The first step in the proposed fusion approach is the histogram matching of one of the images to the other to allow the same dynamic range for both images. The NSCT is used after that to decompose the images to be fused into their coefficients. The MCFO technique is used to determine the optimum decomposition level and the optimum gain parameters for the best fusion of coefficients based on certain constraints. Finally, an additional contrast enhancement process is applied on the fused image to enhance its visual quality and reinforce details. The proposed fusion framework is subjectively and objectively evaluated with different fusion quality metrics including average gradient, local contrast, standard deviation (STD), edge intensity, entropy, peak signal-to-noise ratio, Q ^ab/f, and processing time. Experimental results demonstrate that the proposed optimized NSCT medical image fusion approach based on the MCFO and histogram matching achieves a superior performance with higher image quality, average gradient, edge intensity, STD, better local contrast and entropy, a good quality factor, and much more details in images. These characteristics help for more accurate medical diagnosis in different medical applications.     

Fusion of noisy images based on joint distribution model in dual-tree complex wavelet domain

Source images are frequently corrupted by noise before fusion, which will lead to the quality decline of fused image and the inconvenience for subsequent observation. However, at present, most of the traditional medical image fusion scheme cannot be implemented in noisy environment. Besides, the existing fusion methods scarcely make full use of the dependencies between source images. In this research, a novel fusion algorithm based on the statistical properties of wavelet coefficients is proposed, which incorporates fusion and denoising simultaneously. In the proposed algorithm, the new saliency and matching measures are defined by two distributions: the marginal statistical distribution of single wavelet coefficient fit by the generalized Gaussian Distribution and joint distribution of dual source wavelet coefficients modeled by the anisotropic bivariate Laplacian model. Additionally, the bivariate shrinkage is introduced to develop a noise robust fusion method, and a moment-based parameter estimation applied in the fusion scheme is also exploited in denoising method, which allows to achieve the consistency of fusion and denoising. The experiments demonstrate that the proposed algorithm performs very well on both noisy and noise-free images from multimodal medical datasets (computerized tomography, magnetic resonance imaging, magnetic resonance angiography, etc.), outperforming the conventional methods in terms of both fusion quality and noise reduction.     

Medical image enhancement algorithm based on NSCT and the improved fuzzy contrast

In order to solve the problem of noise amplification, low contrast and image distortion in the process of medical image enhancement, a new algorithm is proposed which combines NSCT (nonsubsampled contourlet transform) and improved fuzzy contrast. The image is decomposed by NSCT. Firstly, linear enhancement method is used in low frequency coefficients; secondly the improved adaptive threshold function is used to deal with the high frequency coefficients. Finally, the improved fuzzy contrast is used to enhance the global contrast and the Laplace operator is used to enhance the details of the medical images. Experimental results show that the proposed algorithm can improve the image visual effects, remove the noise and enhance the details of medical images. © 2015 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 25, 7–14, 2015     

Multimodal medical image fusion based on discrete Tchebichef moments and pulse coupled neural network

Multimodal medical image fusion plays a vital role in clinical diagnoses and treatment planning. In many image fusion methods‐based pulse coupled neural network (PCNN), normalized coefficients are used to motivate the PCNN, and this makes the fused image blur, detail loss, and decreases contrast. Moreover, they are limited in dealing with medical images with different modalities. In this article, we present a new multimodal medical image fusion method based on discrete Tchebichef moments and pulse coupled neural network to overcome the aforementioned problems. First, medical images are divided into equal‐size blocks and the Tchebichef moments are calculated to characterize image shape, and energy of blocks is computed as the sum of squared non‐DC moment values. Then to retain edges and textures, the energy of Tchebichef moments for blocks is introduced to motivate the PCNN with adaptive linking strength. Finally, large firing times are selected as coefficients of the fused image. Experimental results show that the proposed scheme outperforms state‐of‐the‐art methods and it is more effective in processing medical images with different modalities. © 2017 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 27, 57–65, 2017     

FWFusion: Fuzzy Whale Fusion model for MRI multimodal image fusion

Medical treatment and diagnosis require information that is taken from several modalities of images like Magnetic Resonance Imaging (MRI), Computerized Tomography and so on. The information obtained for certain ailments is often incomplete, invisible and lacking in consistent scanner performance. Hence, to overcome these issues in the image modalities, image fusion schemes are developed in the literature. This paper proposes a hybrid algorithm using fuzzy concept and a novel P-Whale algorithm, called Fuzzy Whale Fusion (FWFusion), for the fusion of MRI multimodal images. Two multimodal images from MRI (T1, T1C, T2 and FLAIR) are considered as the source images, which are fed as inputs to a wavelet transform. The transform utilized converts the images into four different bands, which are fused using two newly derived fusion factors, fuzzy fusion and whale fusion, in a weighted function. The proposed P-Whale approach combines Whale Optimization Algorithm (WOA) and Particle Swarm Optimization (PSO) for the effective selection of whale fusion factors. The performance of FWFusion model is compared to those of the existing strategies using Mutual Information (MI), Peak Signal-to-Noise Ratio (PSNR) and Root Mean Squared Error (RMSE), as the evaluation metrics. From the mean performance evaluation, it is observed that the proposed approach can achieve MI of 1.714, RMSE of 1.9 and PSNR of 27.9472.     

A novel brain image enhancement method based on nonsubsampled contourlet transform

In this article, a novel brain image enhancement approach based on nonsubsampled contourlet transform (NSCT) is proposed. First, the image is decomposed into a low‐frequency component and several high‐frequency components by the NSCT; Second, the gamma correction is applied to deal with the low‐frequency sub‐band coefficients, and the adaptive threshold is used to remove the noise of the high‐frequency sub‐bands coefficients; Third, the inverse nonsubsampled contourlet transform is adopted to reconstruct the processed coefficients; Finally, the unsharp filter is used to enhance the reconstructed image. The experimental results demonstrate that the performance of the proposed method is superior to the state‐of‐the‐art algorithms in terms of brain image enhancement.     

An improved multimodal medical image fusion scheme based on hybrid combination of nonsubsampled contourlet transform and stationary wavelet transform

This research proposes an improved hybrid fusion scheme for non-subsampled contourlet transform (NSCT) and stationary wavelet transform (SWT). Initially, the source images are decomposed into different sub-bands using NSCT. The locally weighted sum of square of the coefficients based fusion rule with consistency verification is used to fuse the detailed coefficients of NSCT. The SWT is employed to decompose approximation coefficients of NSCT into different sub-bands. The entropy of square of the coefficients and weighted sum-modified Laplacian is employed as the fusion rules with SWT. The final output is obtained using inverse NSCT. The proposed research is compared with existing fusion schemes visually and quantitatively. From the visual analysis, it is observed that the proposed scheme retained important complementary information of source images in a better way. From the quantitative comparison, it is seen that this scheme gave improved edge information, clarity, contrast, texture, and brightness in the fused image.     

An Efficient Medical Image Deep Fusion Model Based on Convolutional Neural Networks

Medical image fusion is considered the best method for obtaining one image with rich details for efficient medical diagnosis and therapy. Deep learning provides a high performance for several medical image analysis applications. This paper proposes a deep learning model for the medical image fusion process. This model depends on Convolutional Neural Network (CNN). The basic idea of the proposed model is to extract features from both CT and MR images. Then, an additional process is executed on the extracted features. After that, the fused feature map is reconstructed to obtain the resulting fused image. Finally, the quality of the resulting fused image is enhanced by various enhancement techniques such as Histogram Matching (HM), Histogram Equalization (HE), fuzzy technique, fuzzy type Π, and Contrast Limited Histogram Equalization (CLAHE). The performance of the proposed fusion-based CNN model is measured by various metrics of the fusion and enhancement quality. Different realistic datasets of different modalities and diseases are tested and implemented. Also, real datasets are tested in the simulation analysis.     

An Efficient DWT and Intuitionistic Fuzzy Based Multimodality Medical Image Fusion

Image fusion aims to integrate complementary information from multiple modalities into a single image with none distortion and loss of data. Image fusion is important in medical imaging, specifically for the purpose of detecting the tumor and identification of diseases. In this article, completely unique discrete wavelet transform (DWT) and intuitionistic fuzzy sets (IFSs) based fusion method (DWT‐IFS) is proposed. For fusion, initially, all source images are fused using DWT with the average, maximum, and entropy fusion rules. Besides, on the fused image IFS is applied. In the IFS process images are converted into intuitionistic fuzzy images (IFIs) by selecting an optimum value for the parameter in membership, non‐membership, and hesitation degree function using entropy. Then, the resulting IFIs are decomposed into the blocks, and the corresponding blocks of the images are fused using the intersection and union operations of IFS. The efficiency of the proposed DWT‐IFS fusion method is recognized by examining it with other existing methods, such as Averaging (AVG), Principal Component Analysis (PCA), Laplacian Pyramid Approach (LPA), Contrast Pyramid Approach (CPA), Discrete Wavelet Transform (DWT), Morphological Pyramid Approach (MPA), Redundancy Discrete Wavelet Transform (RDWT), Contourlet Transform (CONTRA), and Intuitionistic Fuzzy Set (IFS) using subjective and objective performance evaluation measures. The experimental results reveal that the proposed DWT‐IFS fusion method provides higher quality of information in terms of physical properties and contrast as compared to the existing methods.     

Image speckle noise denoising by a multi-layer fusion enhancement method based on block matching and 3D filtering

In order to improve speckle noise denoising of block matching and 3D filtering (BM3D) method, an image frequency-domain multi-layer fusion enhancement method (MLFE-BM3D) based on nonsubsampled contourlet transform (NSCT) has been proposed. The method designs an NSCT hard threshold denoising enhancement to preprocess the image, then uses fusion enhancement in NSCT domain to fuse the preliminary estimation results of images before and after the NSCT hard threshold denoising, finally, BM3D denoising is carried out with the fused image to obtain the final denoising result. Experiments on natural images and medical ultrasound images show that MLFE-BM3D method can achieve better visual effects than BM3D method, the peak signal to noise ratio (PSNR) of the denoised image is increased by 0.5?dB. The MLFE-BM3D method can improve the denoising effect of speckle noise in the texture region, and still maintain a good denoising effect in the smooth region of the image.     

基于小波系数邻域特征的CT与MR图像融合方法

提出一种基于小波变换的像素级CT,MR医学图像融合方法,利用离散小波变换分别将两幅源图像进行多尺度分解,再用不同的小波系数邻域特征指导高频分量和低频分量的小波系数的融合,低频分量采用邻域方差指导,高频分量采用邻域能量指导,最后根据融合图像的各小波系数重构融合图像.实验表明:不论从主观感受,还是采用信息熵和平均梯度两项指标作为客观定量评价标准,该方法都优于传统的融合方法,获得的融合图像有效地综合了CT与MR图像信息,能够同时清晰地显示脑部骨组织和软组织.     

Computer aided automated detection and classification of brain tumors using CANFIS classification method

The development of abnormal cells in human brain leads to the formation of tumors. This article proposes an efficient approach for brain tumor detection and segmentation using image fusion and co-active adaptive neuro fuzzy inference system (CANFIS) classification method. The brain MRI images are fused and the dual tree complex wavelet transform is applied on the fused image. Then, the statistical features, local ternary pattern features and gray level co-occurrence matrix features. These extracted features are classified using CANFIS classification approach for the classification of source brain MRI image into either normal or abnormal. Further, morphological operations are applied on the abnormal brain MRI image for segmenting the tumor regions. The proposed methodology is evaluated with respect to the performance metrics sensitivity, specificity, positive predictive value, negative predictive value, tumor segmentation accuracy with detection rate. The proposed image fusion based brain tumor detection and classification methodology stated in this article achieves 96.5% of average sensitivity, 97.7% of average specificity, 87.6% of positive predictive value, 96.6% of negative predictive value, and 98.8% of average accuracy.