A multimodal molecular image fusion method based on relative total variation and co-saliency detection

Image fusion can integrate complementary information from multimodal molecular images to provide an informative single result image. In order to obtain a better fusion effect, this article proposes a novel method based on relative total variation and co-saliency detection (RTVCSD). First, only the gray-scale anatomical image is decomposed into a base layer and a texture layer according to the relative total variation; then, the three-channel color functional image is transformed into the luminance and chroma (YUV) color space, and the luminance component Y is directly fused with the base layer of the anatomical image by comparing the co-saliency information; next, the fused base layer is linearly combined with the texture layer, and the obtained fused result is combined with the chroma information U and V of the functional image. Finally, the fused image is obtained by transforming back to the red–green–blue color space. The dataset consists of magnetic resonance imaging (MRI)/positron emission tomography images, MRI/single photon emission computed tomography (SPECT) images, computed tomography/SPECT images, and green fluorescent protein/phase contrast images, each category with 20 image pairs. Experimental results demonstrate that the proposed method RTVCSD outperforms the nine comparison algorithms in terms of visual effects and objective evaluation. RTVCSD well preserves the texture information of the anatomical image and the metabolism or protein distribution information of the functional image.     

Medical image fusion based on laws of texture energy measures in stationary wavelet transform domain

Medical image fusion is widely used in various clinical procedures for the precise diagnosis of a disease. Image fusion procedures are used to assist real-time image-guided surgery. These procedures demand more accuracy and less computational complexity in modern diagnostics. Through the present work, we proposed a novel image fusion method based on stationary wavelet transform (SWT) and texture energy measures (TEMs) to address poor contrast and high-computational complexity issues of fusion outcomes. SWT extracts approximate and detail information of source images. TEMs have the capability to capture various features of the image. These are considered for fusion of approximate information. In addition, the morphological operations are used to refine the fusion process. Datasets consisting of images of seven patients suffering from neurological disorders are used in this study. Quantitative comparison of fusion results with visual information fidelity-based image fusion quality metric, ratio of spatial frequency error, edge information-based image fusion quality metric, and structural similarity index-based image fusion quality metrics proved the superiority. Also, the proposed method is superior in terms of average execution time to state-of-the-art image fusion methods. The proposed work can be extended for fusion of other imaging modalities like fusion of functional image with an anatomical image. Suitability of the fused images by the proposed method for image analysis tasks needs to be studied.     

Multimodal sensor medical image fusion based on mutual‐structure for joint filtering using sparse representation

Multimodal sensor medical image fusion has been widely reported in recent years, but the fused image by the existing methods introduces low contrast information and little detail information. To overcome this problem, the new image fusion method is proposed based on mutual‐structure for joint filtering and sparse representation in this article. First, the source image is decomposed into a series of detail images and coarse images by mutual‐structure for joint filtering. Second, sparse representation is adopted to fuse coarse images and then local contrast is applied for fusing detail images. Finally, the fused image is reconstructed by the addition of the fused coarse images and the fused detail images. By experimental results, the proposed method shows the best performance on preserving detail information and contrast information in the views of subjective and objective evaluations.     

Precise image fusion standardization for separated modalities using dedicated multimodal heart phantom

The standardization of images derived from different medical modalities should be ensured when image fusion brings essential information and hybrid scanners are not available. The aim of this study was to show that precise image fusion standardization can be obtained using special and unique multimodal heart phantom (MHP) which is compatible with all applied diagnostic methods. MHP was designed and constructed according to International Commission on Radiological Protection reports, scanners requirements and personal experience. Three different types of acquisitions were done: physiological perfusion, myocardial ischaemia and intestines artefacts. The measurements were done using different modalities (single photon emission computed tomography (SPECT), positron emission tomography (PET), MRI, CT) as well as hybrid scanners (PET/CT, SPECT/CT). It was shown that MHP can be used not only for improvement of image fusion standardization protocol but also for verification of proper implementation of the fusion protocol in hybrid scanners.     

Fusion of MRI and CT images using guided image filter and image statistics

In medical imaging using different modalities such as MRI and CT, complementary information of a targeted organ will be captured. All the necessary information from these two modalities has to be integrated into a single image for better diagnosis and treatment of a patient. Image fusion is a process of combining useful or complementary information from multiple images into a single image. In this article, we present a new weighted average fusion algorithm to fuse MRI and CT images of a brain based on guided image filter and the image statistics. The proposed algorithm is as follows: detail layers are extracted from each source image by using guided image filter. Weights corresponding to each source image are calculated from the detail layers with help of image statistics. Then a weighted average fusion strategy is implemented to integrate source image information into a single image. Fusion performance is assessed both qualitatively and quantitatively. Proposed method is compared with the traditional and recent image fusion methods. Results showed that our algorithm yields superior performance.     

Improving medical image fusion method using fuzzy entropy and nonsubsampling contourlet transform

Many types of medical images must be fused, as single‐modality medical images can only provide limited information due to the imaging principles and the complexity of human organ structures. In this paper, a multimodal medical image fusion method that combines the advantages of nonsubsampling contourlet transform (NSCT) and fuzzy entropy is proposed to provide a basis for clinical diagnosis and improve the accuracy of target recognition and the quality of fused images. An image is initially decomposed into low‐ and high‐frequency subbands through NSCT. The corresponding fusion rules are adopted in accordance with the different characteristics of the low‐ and high‐frequency components. The membership degree of low‐frequency coefficients is calculated. The fuzzy entropy is also computed and subsequently used to guide the fusion of coefficients to preserve image details. High‐frequency components are fused by maximizing the regional energy. The final fused image is obtained by inverse transformation. Experimental results show that the proposed method achieves good fusion effect based on the subjective visual effect and objective evaluation criteria. This method can also obtain high average gradient, SD, and edge preservation and effectively retain the details of the fused image. The results of the proposed algorithm can provide effective reference for doctors to assess patient condition.     

Multi-focus image fusion based on the non-subsampled contourlet transform

In this paper, a new image fusion algorithm based on non-subsampled contourlet transform (NSCT) is proposed for the fusion of multi-focus images. The selection of different subband coefficients obtained by the NSCT decomposition is critical to image fusion. So, in this paper, firstly, original images are decomposed into different frequency subband coefficients by NSCT. Secondly, the selection of the low-frequency subband coefficients and the bandpass directional subband coefficients is discussed in detail. For the selection of the low-frequency subband coefficients, the non-negative matrix factorization (NMF) method is adopted. For the selection of bandpass directional subband coefficients, a regional cross-gradient method that selects the coefficients according to the minimum of the regional cross-gradient is proposed. Finally, the fused image is obtained by performing the inverse NSCT on the combined coefficients. The experimental results show that the proposed fusion algorithm can achieve significant results in getting a new image where all parts are sharp.     

Medical image fusion based on nuclear norm minimization

Medical image fusion plays an important role in diagnosis and treatment of diseases such as image‐guided radiotherapy and surgery. Although numerous medical image fusion methods have been proposed, most approaches have not touched the low rank nature of matrix formed by medical image, which usually lead to fusion image distortion and image information loss. These methods also often lack universality when dealing with different kinds of medical images. In this article, we propose a novel medical image fusion to overcome aforementioned issues on existing methods with the aid of low rank matrix approximation with nuclear norm minimization (NNM) constraint. The workflow of our method is described as: firstly, nonlocal similar patches across the medical image are searched by block matching for local patch in source images. Second, a fused matrix is stacking by shared nonlocal similarity patches, then the low rank matrix approximation methods under nuclear norm minimization can be used to recover low rank feature of fused matrix. Finally, fused image can be gotten by aggregating all the fused patches. Experimental results show that the proposed method is superior to other methods in both subjectively visual performance and objective criteria. © 2015 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 25, 310–316, 2015     

Medical image fusion based on improved sum‐modified‐Laplacian

Sum‐modified‐Laplacian (SML) plays an important role in medical image fusion. However, fused rules based on larger SML always lead to fusion image distortion in transform domain image fusion or image information loss in spatial domain image fusion. Combined with average filter and median filter, a new medical image fusion method based on improved SML (ISML) is proposed. First, a basic fused image is gained by ISML, which is used for evaluation of the selection map of medical images. Second, difference images can be obtained by subtracting average image of all sources of medical images. Finally, basic fused image can be refined by difference images. The algorithm can both preserve the information of the source images well and suppress pixel distortion. Experimental results demonstrate that the proposed method outperforms the state‐of‐the‐art medical image fusion methods. © 2015 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 25, 206–212, 2015     

Statistical correlative model in the multimodal fusion of brain images

Fusing multimodal medical images into an integrated image, providing more details and rich information thereby facilitating medical diagnosis and therapy. Most of the existing multiscale-based fusion methods ignore the correlations between the decomposition coefficients and lead to incomplete fusion results. A novel contextual hidden Markov model (CHMM) is proposed to construct the statistical model of contourlet coefficients. First, the pair brain images are decomposed into multiscale, multidirectional, and anisotropic subbands with a contourlet transform. Then the low-frequency components are fused with the choose-max rule. For the high-frequency coefficients, the CHMM is learned with the EM algorithm, and incorporate with a novel fuzzy entropy-based context, building the fuzzy relationships among these coefficients. Finally, the fused brain image is obtained by using the inverse contourlet transform. Fusion experiments on several multimodal brain images show the superiority of the proposed method in terms of both visual quality and some widely used objective measures.     

A novel multi-modal medical image fusion method based on shift-invariant shearlet transform

Abstract

Medical image fusion plays an important role in clinical applications, such as image-guided surgery, image-guided radiotherapy, non-invasive diagnosis and treatment planning. Shearlet is a novel multi-scale geometric analysis (MGA) tool proposed recently. In order to overcome the drawback of the shearlet-based fusion methods that the pseudo-Gibbs phenomenon is easily caused around the singularities of the fused image, a new multi-modal medical image fusion method is proposed in shift-invariant shearlet transform domain. First, the original images are decomposed into lowpass sub-bands and highpass sub-bands; then, the lowpass sub-bands and high sub-bands are combined according to the fusion rules, respectively. All the operations are performed in shift-invariant shearlet domain. The final fused image is obtained by directly applying inverse shift-invariant shearlet transform to the fused lowpass sub-bands and highpass sub-bands. Experimental results demonstrate that the proposed method can not only suppress the pseudo-Gibbs phenomenon efficiently, but perform better than the popular wavelet transform-based method, contourlet transform-based method and non-subsampled contourlet transform-based method.     

一种基于小波变换的多聚焦图像融合方法

提出了一种基于低频系数局部区域梯度信息的多分辨率图像融合方法。根据局部梯度信息对源图像的小波低频系数进行选择,获取融合图像的对应低频系数。依照平均误差、峰值信噪比、均方根误差以及偏差度、熵等评价标准,将该方法的多聚焦图像融合效果与其他三种常用低频系数融合方法的效果进行了比较。实验结果表明,该方法获得的大部分评价指标都优于其他三种方法,且其最佳小波分解层数为2层,而其他三种方法的最佳小波分解层数为5层。最佳小波分解层数越少,图像融合的计算量越小。该方法在减少计算量的同时,提高了融合质量。     

Modified Global Flower Pollination Algorithm‐based image fusion for medical diagnosis using computed tomography and magnetic resonance imaging

Recently, the computed tomography (CT) and magnetic resonance imaging (MRI) medical image fusion have turned into a challenging issue in the medical field. The optimal fused image is a significant component to detect the disease easily. In this research, we propose an iterative optimization approach for CT and MRI image fusion. Initially, the CT and MRI image fusion is subjected to a multilabel optimization problem. The main aim is to minimize the data and smoothness cost during image fusion. To optimize the fusion parameters, the Modified Global Flower Pollination Algorithm is proposed. Here, six sets of fusion images with different experimental analysis are evaluated in terms of different evaluation metrics such as accuracy, specificity, sensitivity, SD, structural similarity index, feature similarity index, mutual information, fusion quality, and root mean square error (RMSE). While comparing to state‐of‐art methods, the proposed fusion model provides best RMSE with higher fusion performance. Experiments on a set of MRI and CT images of medical data set show that the proposed method outperforms a very competitive performance in terms of fusion quality.     

Robust medical image registration based on phase congruency and regional mutual information

Abstract

In this paper, a new approach of multi-modality image registration is represented with not only image intensity, but also features describing image structure. There are two novelties in the proposed method. First, instead of standard mutual information based on joint intensity histogram, a graph-based implementation of multi-dimensional regional mutual information is employed, which allows neighbourhood information to be taken into account. Second, a new feature image is obtained by means of phase congruency, which is invariant to brightness or contrast changes. By incorporating these features and intensity into regional mutual information, we can combine aspects of both structural and neighbourhood information together, which offers a more robust and a high level of registration accuracy that is essential in application to the medical domain.     

基于多策略的多聚焦图像融合方法

针对多聚焦图像融合,在小波变换基础上提出了一种多策略的融合方法.图像小波变换后的低频分量采用基于清晰度的方法,清晰度评价函数选用八邻域拉普拉斯算子和;而高频分量采用基于空间频率的方法,并且只计算与高频方向相一致的空间频率,减小了计算量.融合效果评价除使用传统方法的熵、交叉熵外,还使用了基于三个方向(水平、垂直和对角)的空间频率和通用的主观与客观相结合的新方法.实验结果表明,本文方法得到的融合图像效果最好,优于文中的其它算法,同时本文使用的评价方法也是适用和有效的.     

基于多目标粒子群算法的多传感器图像融合

针对图像融合中参数优化的问题,提出了一种基于多目标粒子群优化算法的多传感器图像融合方法。首先采用非采样Contourlet变换(NSCT)对源图像进行多尺度、多方向分解;然后选取图像融合的客观评价指标为优化目标函数,采用多目标粒子群优化算法对低频系数的融合参数进行优化,带通方向子带系数采用取绝对值最大的融合规则;最后通过NSCT逆变换得到融合图像。分别对多聚焦图像融合和红外与可见光图像进行融合实验,并对融合图像进行主客观评价,实验结果表明,得到的融合图像具有较好的主观视觉效果和客观评价指标。     

Optimal multi-scale geometric fusion based on non-subsampled contourlet transform and modified central force optimization

In the current era of technological development, medical imaging plays an important part in several applications of medical diagnosis and therapy. This requires more precise images with much more details and information for correct medical diagnosis and therapy. Medical image fusion is one of the solutions for obtaining much spatial and spectral information in a single image. This article presents an optimization-based contourlet image fusion approach in addition to a comparative study for the performance of both multi-resolution and multi-scale geometric effects on fusion quality. An optimized multi-scale fusion technique based on the Non-Subsampled Contourlet Transform (NSCT) using the Modified Central Force Optimization (MCFO) and local contrast enhancement techniques is presented. The first step in the proposed fusion approach is the histogram matching of one of the images to the other to allow the same dynamic range for both images. The NSCT is used after that to decompose the images to be fused into their coefficients. The MCFO technique is used to determine the optimum decomposition level and the optimum gain parameters for the best fusion of coefficients based on certain constraints. Finally, an additional contrast enhancement process is applied on the fused image to enhance its visual quality and reinforce details. The proposed fusion framework is subjectively and objectively evaluated with different fusion quality metrics including average gradient, local contrast, standard deviation (STD), edge intensity, entropy, peak signal-to-noise ratio, Q ^ab/f, and processing time. Experimental results demonstrate that the proposed optimized NSCT medical image fusion approach based on the MCFO and histogram matching achieves a superior performance with higher image quality, average gradient, edge intensity, STD, better local contrast and entropy, a good quality factor, and much more details in images. These characteristics help for more accurate medical diagnosis in different medical applications.     

基于结构相似性的图像质量评价方法

分析现有图像质量评价方法特点,针对基于失真敏感度的质量评价方法的局限性,提出了一种新的基于结构相似度的图像质量评价方法,从亮度、对比度、结构三个子方面得到一个总的相似性度量作为质量客观评价标准.该方法充分考虑了图像的结构信息和人类视觉的特性,从图像内容的理解功能出发,通过数学建模估算出人眼对图像质量的主观视觉感受,使结构相似性计算模型符合图像处理应用的本质.通过理论推导和算法验证,该方法可以为选择图像压缩算法和评价图像质量提供依据.将采用压缩算法SPIHT编解码后的图像与传统的峰值信噪比方法评价图像相比,实验表明,本文提出的算法是一种更为有效的图像质量评价方法.     

Morphologic field morphing: Contour model-guided image interpolation

An interpolation method using contours of organs as the control parameters is proposed to recover the intensity information in the physical gaps of serial cross-sectional images. In our method, contour models are used to generate the control lines required for the warping algorithm. Contour information derived from this contour model-based segmentation process is processed and used as the control parameters to warp the corresponding regions in both input images into compatible shapes. In this way, the reliability of establishing the correspondence among different segments of the same organs is improved and the intensity information for the interpolated intermediate slices can be derived more faithfully. To improve the efficiency for calculating the image warp in the field morphing process, a hierarchic decomposition process is proposed to localize the influence of each control line segment. In comparison with the existing intensity interpolation algorithms that only search for corresponding points in a small physical neighborhood, this method provides more meaningful correspondence relationships by warping regions in images into similar shapes before resampling to account for significant shape differences. Several sets of experimental result are presented to show that this method generates more realistic and less blurred interpolated images, especially when the shape difference of corresponding contours is significant. © 1997 John Wiley & Sons, Inc. Int J Imaging Syst Technol, 8, 480–490, 1997     

Multifocus noisy image fusion using contourlet transform

Abstract

Image fusion is a challenging area of research with a variety of applications. The process of image fusion collects information from different sources and combines them in a single composite image. The composite fused image can better describe the scene than any of the source images. In this paper, we have proposed a method for noisy image fusion in contourlet domain. The proposed method works equally well for fusion of noise free images. Contourlet transform is a multiscale, multidirectional transform with various aspect ratios. These properties make it more suitable for image fusion than other conventional transforms. In the proposed work, the fusion algorithm is combined with a denoising algorithm to reverse the effect of noise. In the proposed method, we have used a level dependent threshold that is based on standard deviation of contourlet coefficients, mean and median of the absolute contourlet coefficients. Experimental results demonstrate that the proposed method performs well in the presence of different types of noise. Performance of the proposed method is compared with principal components analysis and sharp fusion based methods as well as other fusion methods based on variants of wavelet transform like dual tree complex wavelet transform, discrete wavelet transform, lifting wavelet transform, multiwavelet transform, stationary wavelet transform and pyramid transform using six standard quantitative quality metrics (entropy, standard deviation, edge strength, fusion factor, sharpness and peak signal to noise ratio). The combined qualitative and quantitative evaluation of the experimental results shows that the proposed method performs better than other methods.