形态学滤波和改进PCNN的NSST域多光谱与全色图像融合

目的 全色图像的空间细节信息增强和多光谱图像的光谱信息保持通常是相互矛盾的，如何能够在这对矛盾中实现最佳融合效果一直以来都是遥感图像融合领域的研究热点与难点。为了有效结合光谱信息与空间细节信息，进一步改善多光谱与全色图像的融合质量，提出一种形态学滤波和改进脉冲耦合神经网络（PCNN）的非下采样剪切波变换（NSST）域多光谱与全色图像融合方法。方法 该方法首先分别对多光谱和全色图像进行非下采样剪切波变换；对二者的低频分量采用形态学滤波和高通调制框架（HPM）进行融合，将全色图像低频子带的细节信息注入到多光谱图像低频子带中得到融合后的低频子带；对二者的高频分量则采用改进脉冲耦合神经网络的方法进行融合，进一步增强融合图像中的空间细节信息；最后通过NSST逆变换得到融合图像。结果 仿真实验表明，本文方法得到的融合图像细节信息清晰且光谱保真度高，视觉效果上优势明显，且各项评价指标与其他方法相比整体上较优。相比于5种方法中3组融合结果各指标平均值中的最优值，清晰度和空间频率分别比NSCT-PCNN方法提高0.5%和1.0%，光谱扭曲度比NSST-PCNN方法降低4.2%，相关系数比NSST-PCNN方法提高1.4%，信息熵仅比NSST-PCNN方法低0.08%。相关系数和光谱扭曲度两项指标的评价结果表明本文方法相比于其他5种方法能够更好地保持光谱信息，清晰度和空间频率两项指标的评价结果则展示了本文方法具有优于其他对比方法的空间细节注入能力，信息熵指标虽不是最优值，但与最优值非常接近。结论 分析视觉效果及各项客观评价指标可以看出，本文方法在提高融合图像空间分辨率的同时，很好地保持了光谱信息。综合来看，本文方法在主观与客观方面均具有优于亮度色调饱和度（IHS）法、主成分分析（PCA）法、基于非负矩阵分解（CNMF）、基于非下采样轮廓波变换和脉冲耦合神经网络（NSCT-PCNN）以及基于非下采样剪切波变换和脉冲耦合神经网络（NSST-PCNN）5种经典及现有流行方法的融合效果。

Fusion of multispectral and panchromatic images via morphological filter and improved PCNN in NSST domain

Objective Various remote sensing sensors presently exist, and multisource remote sensing images, such as multispectral (MS) and panchromatic (PAN) images, can be acquired. MS images, which have rich spectral information and low spatial resolution, cannot meet the remote sensing application demand. Correspondingly, PAN images have more spatial details and higher spatial resolutions. The significance of MS and PAN image fusion is that it improves the spatial resolution of MS images while maintaining original spectral information. It also combines target shape and the structural characteristics of PAN images and the spectral information of MS images to provide great interpretation capability and reliable results, as well as enhances the classification and identification precision of objects. However, the spatial resolution enhancement of PAN images and the spectral information maintenance of MS images are usually contradictory. How to acquire a high fusion performance in the contradictions has always been a popular and difficult point in the research field of remote sensing image fusion and has an extensive prospect in research and application. In this study, a fusion method based on morphological filter and improved pulse-coupled neural network (PCNN) in a non-subsampled shearlet transform (NSST) domain is proposed to improve the fusion quality of MS and PAN images by combining spectral information with spatial details efficiently. Method The proposed method is conducted on MS and PAN images that have been accurately registered. First, the PAN and MS images are decomposed by NSST to obtain low- and high-frequency sub-band coefficients. Second, the low-frequency sub-bands, which are approximate sub-graphs of the original image and inherit the overall characteristics, still have some edges and detailed information. The fusion rule of low-frequency coefficients based on morphological filtering and high-pass modulation (HPM) scheme is proposed. The morphological half-gradient operator is used to extract the details of the low-frequency sub-bands of the PAN image owing to its preliminary encouraging fusion results on remote sensing images. The low-resolution PAN sub-band image can be obtained by morphological filtering, and the detailed PAN sub-band image is estimated by subtracting the low-resolution PAN sub-band image from the PAN sub-band image equalized with histogram on the basis of the MS sub-band image. The spatial details are then injected into the low-frequency sub-band of the MS image through the HPM scheme. For the fusion of high-frequency sub-bands, an improved PCNN is taken to enhance spatial detail information. Existing PCNN models usually adopt a hard-limiting function as output, and the firing output is 0 or 1, which cannot reflect the amplitude difference of the synchronous pulse excitation efficiently. At a point, a soft-limiting sigmoid function is adopted to calculate the firing output amplitude during the iterations, and the decision matrix for high-frequency coefficient selection can be achieved by summing up the firing output amplitude in the iterative process. Then, the fusion low- and high-frequency coefficients are reconstructed with the inverse NSST to obtain the final fusion image. Result A series of simulation experiments is conducted to verify the superiority and validity of the proposed fusion method. Three groups of QuickBird remote sensing images are utilized to test the proposed method. The performance evaluation of the fusion methods includes the subjective visual effect and objective standard evaluation. Visual analysis is the most immediate detection method. Five objective evaluation indicators, namely, image clarity, information entropy, correlation coefficient, spatial frequency, and spectral distortion, are selected to evaluate the fusion results quantitatively and objectively. Experimental results show that the proposed method has obvious advantages in the fusion effect. The subjective visual effect of the proposed method is obviously better than those of the other five methods. Details such as image textures and edges are clear, and the spectral information is maintained efficiently. Compared with the other fusion methods, the proposed method also has great superiority on the objective evaluation indicators. The average values of the five indicators for three bands are calculated, four of which are the best among the comparison methods. The average value of three groups of images are also calculated. Compared with the best indicator of the other five methods, the image clarity and spatial frequency of our method are improved by 0.5% and 1.0%, respectively, compared with the NSCT-PCNN method. Our spectral distortion is 4.2% lower than that of the NSST-PCNN method. Our correlation coefficient is 1.4% higher than that of NSST-PCNN, and the information entropy is only 0.08% lower than the best value from NSST-PCNN. The results of the correlation coefficient and spectral distortion demonstrate that the proposed method maintains better spectral information than do the other five methods. Results of the image clarity and spatial frequency show that the proposed method has an excellent capability of detailed information injection, and only the image clarity of B band in group 2 is poor. The information entropy is approximate to the best result. Conclusion A fusion method of MS and PAN images based on morphological operator and improved PCNN in NSST domain is proposed. We present the fusion rules for different frequency bands according to the NSST decomposition of the original MS and PAN images. Low-frequency coefficient fusion rule based on morphological half-gradient filtering and the HPM scheme and high-frequency coefficient fusion rule based on the improved PCNN are designed. A real satellite dataset is employed for the performance evaluation of the proposed method. The analysis indicates that our method can improve the spatial resolution and maintain the spectral information of fusion results. In general, the proposed method is superior to the traditional methods and some current popular fusion methods from the overall effect of visual aspects and objective indicators.