A Model-Based Approach to Multiresolution Fusion in Remotely Sensed Images

In this paper, a model-based approach to multiresolution fusion of remotely sensed images is presented. Given a high spatial resolution panchromatic (Pan) image and a lowspatial resolution multispectral (MS) image acquired on the same geographical area, the presented method aims to enhance the spatial resolution of the MS image to the resolution of the Pan observation. The proposed fusion technique utilizes the spatial correlation of each of the high-resolution MS channels by using an autoregressive (AR) model, whose parameters are learnt from the analysis of the Pan data. Under the assumption that the parameters of the AR model for the Pan image are the same as those that represent the MS images due to spectral correlation, the proposed technique exploits the learnt parameter values in the context of a proper regularization technique to estimate the high spatial resolution fields for the MS bands. This results in a combination of the spectral characteristics of the low-resolution MS data with the high spatial resolution of the Pan image. The main advantages of the proposed technique are: 1) unlike standard methods proposed in the literature, it requires no registration between the Pan and the MS images; 2) it models effectively the texture of the scene during the fusion process; 3) it shows very small spectral distortion (as it is less affected, compared to standard methods, by the specific digital numbers of pixels in the Pan image, since it exploits the learnt parameters from the Pan image rather than the actual Pan digital numbers for fusion); and 4) it can be used in critical situations in which the Pan and the MS images are acquired (also by different sensors) in slightly different areas. Quantitative experimental results obtained using Landsat-7 Enhanced Thematic Mapper Plus (ETM+) and Quickbird images point out the effectiveness of the proposed method.     

基于细节信息提取的全色与多光谱图像融合方法

全色（Panchromatic, Pan）图像与多光谱（Multi-spectral, MS）图像融合的目的是生成具有高空间分辨率的多光谱图像。为了进一步提升融合图像的质量,提出一种基于细节信息提取的融合方法。首先,使用滚动引导滤波器与差值运算分别获取Pan与MS的高频分量。其次,采用自适应强度-色度-饱和度（AdaptiveIntensity-Hue-Saturation,AIHS）变换处理MS的高频分量与经像素显著性检测后Pan的高频分量,生成对应的强度分量（Intensity,Ⅰ）,再将Pan与Ⅰ作差值运算获取细节图像。接着,采用引导滤波器计算Pan与MS的高频分量的差值,得到残差图像。最后,利用最速下降法将细节图像与残差图像注入到原始的MS图像中获得最终融合结果。实验结果表明,本文所提算法得到的融合图像能够取得较好的主观视觉效果,且客观定量评价指标较优。     

Estimation of the Number of Decomposition Levels for a Wavelet-Based Multiresolution Multisensor Image Fusion

The wavelet-based scheme for the fusion of multispectral (MS) and panchromatic (PAN) imagery has become quite popular due to its ability to preserve the spectral fidelity of the MS imagery while improving its spatial quality. This is important if the resultant imagery is used for automatic classification. Wavelet-based fusion results depend on the number of decomposition levels applied in the wavelet transform. Too few decomposition levels result in poor spatial quality fused images. On the other hand, too many levels reduce the spectral similarity between the original MS and the pan-sharpened images. If the shift-invariant wavelet transform is applied, each excessive decomposition level results in a large computational penalty. Thus, the choice of the number of decomposition levels is significant. In this paper, PAN and MS image pairs with different resolution ratios were fused using the shift-invariant wavelet transform, and the optimal decomposition levels were determined for each resolution ratio. In general, it can be said that the fusion of images with larger resolution ratios requires a higher number of decomposition levels. This paper provides the practitioner an understanding of the tradeoffs associated with the computational demand and the spatial and spectral quality of the wavelet-based fusion algorithm as a function of the number of decomposition levels     

Generalized IHS‐Based Satellite Imagery Fusion Using Spectral Response Functions

Image fusion is a technical method to integrate the spatial details of the high‐resolution panchromatic (HRP) image and the spectral information of low‐resolution multispectral (LRM) images to produce high‐resolution multispectral images. The most important point in image fusion is enhancing the spatial details of the HRP image and simultaneously maintaining the spectral information of the LRM images. This implies that the physical characteristics of a satellite sensor should be considered in the fusion process. Also, to fuse massive satellite images, the fusion method should have low computation costs. In this paper, we propose a fast and efficient satellite image fusion method. The proposed method uses the spectral response functions of a satellite sensor; thus, it rationally reflects the physical characteristics of the satellite sensor to the fused image. As a result, the proposed method provides high‐quality fused images in terms of spectral and spatial evaluations. The experimental results of IKONOS images indicate that the proposed method outperforms the intensity‐hue‐saturation and wavelet‐based methods.     

一种基于非采样Contourlet变换的遥感图像融合算法

为使融合后的多光谱图像尽可能保持原多光谱图像光谱特性的同时提高空间质量,提出了一种基于非采样Contourlet变换（NSCT）和多尺度边缘检测的融合算法。介绍了非采样Contourlet变换和多尺度边缘检测;设计了基于多尺度边缘检测、直接替代的高频、低频子带融合规则;用QuickBird卫星高分辨率遥感图像进行仿真实验。实验结果表明该算法能够在保持光谱信息的同时注入更丰富的空间细节信息,优于传统的Wavelet变换法和Contourlet变换法。     

联合多流融合和多尺度学习的卷积神经网络遥感图像融合方法

为尽可能保持原始低分辨率多光谱(LRMS)图像光谱信息的同时,显著提高融合后的多光谱图像的空间分辨率,该文提出一种联合多流融合和多尺度学习的卷积神经网络遥感图融合方法。首先将原始MS图像输入频谱特征提取子网得到其光谱特征,然后分别将通过梯度算子处理全色图像得到的梯度信息和通过卷积后的全色图像与得到的光谱特征图在通道上拼接输入到具有多流融合架构的金字塔模块进行图像重构。金字塔模块由多个骨干网络组成,可以在不同的空间感受野下进行特征提取,能够多尺度学习图像信息。最后,构建空间光谱预测子网融合金字塔模块输出的高级特征和网络前端的低级特征得到具有高空间分辨率的MS图像。结合WorldView-3卫星获取的图像进行实验,结果表明,所提方法生成的融合图像在主观目视检验和客观评价指标上都优于大多先进的遥感图像融合方法。     

基于联合卷积分析与合成稀疏表示的遥感图像分量替换融合方法

针对遥感图像融合中传统分量替换方法光谱失真严重问题,提出了一种基于联合卷积分析与合成稀疏表示的改进分量替换融合方法.与传统分量替换方法不同,该方法旨在改进融合过程中空间细节信息提取和注入策略,以生成具有更高光谱与空间质量的遥感图像.首先利用联合卷积分析与合成稀疏表示算法分别对强度分量和直方图匹配后的全色图像进行分解,获...     

Multi-resolution analysis techniques and nonlinear PCA for hybrid pansharpening applications

Hyperspectral images have a higher spectral resolution (i.e., a larger number of bands covering the electromagnetic spectrum), but a lower spatial resolution with respect to multispectral or panchromatic acquisitions. For increasing the capabilities of the data in terms of utilization and interpretation, hyperspectral images having both high spectral and spatial resolution are desired. This can be achieved by combining the hyperspectral image with a high spatial resolution panchromatic image. These techniques are generally known as pansharpening and can be divided into component substitution (CS) and multi-resolution analysis (MRA) based methods. In general, the CS methods result in fused images having high spatial quality but the fused images suffer from spectral distortions. On the other hand, images obtained using MRA techniques are not as sharp as CS methods but they are spectrally consistent. Both substitution and filtering approaches are considered adequate when applied to multispectral and PAN images, but have many drawbacks when the low-resolution image is a hyperspectral image. Thus, one of the main challenges in hyperspectral pansharpening is to improve the spatial resolution while preserving as much as possible of the original spectral information. An effective solution to these problems has been found in the use of hybrid approaches, combining the better spatial information of CS and the more accurate spectral information of MRA techniques. In general, in a hybrid approach a CS technique is used to project the original data into a low dimensionality space. Thus, the PAN image is fused with one or more features by means of MRA approach. Finally the inverse projection is used to obtain the enhanced image in the original data space. These methods, permit to effectively enhance the spatial resolution of the hyperspectral image without relevant spectral distortions and on the same time to reduce the computational load of the entire process. In particular, in this paper we focus our attention on the use of Nonlinear Principal Component Analysis (NLPCA) for the projection of the image into a low dimensionality feature space. However, if on one hand the NLPCA has been proved to better represent the intrinsic information of hyperspectral images in the feature space, on the other hand an analysis of the impact of different fusion techniques applied to the nonlinear principal components in order to define the optimal framework for the hybrid pansharpening has not been carried out yet. More in particular, in this paper we analyze the overall impact of several widely used MRA pansharpening algorithms applied in the nonlinear feature space. The results obtained on both synthetic and real data demonstrate that an accurate selection of the pansharpening method can lead to an effective improvement of the enhanced hyperspectral image in terms of spectral quality and spatial consistency, as well as a strong reduction in the computational time.     

Multi-spectral and hyperspectral image fusion using 3-D wavelet transform

Image fusion is performed between one band of multi-spectral image and two bands of hyperspectral image to produce fused image with the same spatial resolution as source multi-spectral image and the same spectral resolution as source hyperspeetral image. According to the characteristics and 3-Dimensional （3-D） feature analysis of multi-spectral and hyperspectral image data volume, the new fusion approach using 3-D wavelet based method is proposed. This approach is composed of four major procedures： Spatial and spectral resampling, 3-D wavelet transform, wavelet coefficient integration and 3-D inverse wavelet transform. Especially, a novel method, Ratio Image Based Spectral Resampling （RIBSR）method, is proposed to accomplish data resampling in spectral domain by utilizing the property of ratio image. And a new fusion rule, Average and Substitution （A＆S） rule, is employed as the fusion rule to accomplish wavelet coefficient integration. Experimental results illustrate that the fusion approach using 3-D wavelet transform can utilize both spatial and spectral characteristics of source images more adequately and produce fused image with higher quality and fewer artifacts than fusion approach using 2-D wavelet transform. It is also revealed that RIBSR method is capable of interpolating the missing data more effectively and correctly, and A＆S rule can integrate coefficients of source images in 3-D wavelet domain to preserve both spatial and spectral features of source images more properly.     

The Enriched Feature Enhancement Technique for Satellite Image Based on Transforms Using PCNN

The features of the satellite images can be improved by fusing or combining two images with complementary property. By fusing these two images the spatial property of the resultant image is improved. Satellite images are one of the agents that give the features of the earth’s surface. Processing these satellite images will provide more geographical information hidden in the images. This research paper have an detailed insight study of two types of the satellite images one is Panchromatic (PAN) and other Multispectral (MS). The PAN image with high spatial resolution and MS image with spectral resolution are fused to get better resultant output. For fusion process Nonsubsampled Contour let Transform is used to decompose the images into low and high frequency values. Pulse Coupled Neural Network is used to motivate the low frequency pixel and Morphological filter is applied to the edge detected image for finding the features in the images. This is an real time transformations which will give better results in SAR image processing, video processing, stereo based reconstruction of depth and width of the features present in the image.

     

Image Fusion Processing for IKONOS 1-m Color Imagery

Many image fusion techniques have been developed. However, most existing fusion processes produce color distortion in 1-m fused IKONOS images due to nonsymmetrical spectral responses of IKONOS imagery. Here, we proposed a fusion process to minimize this spectral distortion in IKONOS 1-m color images. The 1-m fused image is produced from a 4-m multispectral (MS) and 1-m panchromatic (PAN) image, maintaining the relations of spectral responses between PAN and each band of the MS images. To obtain this relation, four spectral weighting parameters are added with the pixel value of each band of the original MS image. Then, each pixel value is updated using a steepest descent method to reflect the maximum spectral response on the fused image. Comparison among the proposed technique and existing processes [intensity hue saturation (IHS) image fusion, Brovey transform, principal component analysis, fast IHS image fusion] has been done. Our proposed technique has succeeded to generate 1-m fused images where spectral distortion has been reduced significantly, although some block distortions appeared at the edge of the fused images. To remove this block distortion, we also proposed a sharpening process using a wavelet transform, which removed block distortion without significant change in the color of the entire image.     

自适应多尺度几何分析的全色和多光谱图像融合方法研究

为了利用全色和多光谱图像融合得到一幅空间分辨率较高和光谱信息丰富的遥感图像.结合窗口空间频率绝对值最大原则的高频条带波系数融合规则,提出一种基于自适应多尺度几何分析变换的融合方法.利用Landsat-7数据进行试验,得到一幅空间分辨率和光谱信息都较好的融合图像.和轮廓波方法、IHS、小波变换方法进行比较,本方法提高融合图像的质量,图像的边缘细节更明显清晰.     

基于Contourlet系数局部特征的选择性遥感图像融合算法

为了使融合后的多光谱图像在显著提高空间分辨率的同时,尽可能多地保持原始多光谱特性,提出了一种基于Contourlet变换系数局部特征的选择性遥感图像融合方法。根据多光谱和全色图像融合过程中Contourlet变换后的低频和高频部分融合目的的不同,对得到的近似和各层各方向的细节分量分别运用窗口邻域移动模板逐一计算相应区域Contourlet系数阵的不同局部特征量,然后选择适当的准则,对图像的近似和细节分量分别应用不同的策略在Contourlet系数域内进行选择性融合,通过Contourlet和亮度-色调-饱和度(IHS)逆变换得到融合的高分辨率多光谱图像。采用Landsat TM多光谱和SPOT全色图像进行的融合实验结果表明:提出的算法在显著提高空间分辨率的同时,又能很好地保持原始图像的光谱特征,并优于传统的融合方法。     

Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images

In this paper, we propose an optimum algorithm, in the minimum mean-square-error (mmse) sense, for panchromatic (Pan) sharpening of very high resolution multispectral (MS) images. The solution minimizes the squared error between the original MS image and the fusion result obtained by spatially enhancing a degraded version of the MS image through a degraded version, by the same scale factor, of the Pan image. The fusion result is also optimal at full scale under the assumption of invariance of the fusion parameters across spatial scales. The following two versions of the algorithm are presented: a local mmse (lmmse) solution and a fast implementation which globally optimizes the fusion parameters with a moderate performance loss with respect to the lmmse version. We show that the proposed method is computationally practical, even in the case of local optimization, and it outperforms the best state-of-the-art Pan-sharpening algorithms, as resulted from the IEEE Data Fusion Contest 2006, on true Ikonos and QuickBird data and on simulated Pleiades data.     

Practical remote sensing image fusion method based on guided filter and improved SML in the NSST domain

Due to the different characteristics of image modality, the panchromatic (PAN) and multispectral (MS) images include complementary and redundancy information in the spatial and spectral resolutions. Image fusion is an effective way to integrate the source PAN and MS images to obtain high-resolution MS image. In this paper, a novel remote sensing image fusion scheme in non-subsample Shearlet transform (NSST) domain is presented. An enhancement strategy is designed to solve the insufficiency of spatial detail in multiresolution analysis (MRA)-based methods after the intensity–hue–saturation (IHS) color space transform. Then, in the NSST fusion process, a guided filter-based low-frequency coefficient fusion rule and an improved sum-modified-Laplacian (SML)-based high-frequency coefficient fusion rule are proposed. The final fused image can be obtained through the inverse NSST transform and inverse IHS transform. Two different groups of satellite dataset are utilized to evaluate the fusion performance. The experiment results demonstrate that the proposed approach can achieve more spatial details and less spectral distortion compared with the existing methods regarding both the visual quality and the objective measurements.     

一种基于小波变换的可调节遥感影像融合方法

光谱保持和高分辨率保留是影像融合的两个重要问题,不同的应用可能对融合的结果有不同的要求．提出了一种基于小波变换的可调节遥感影像融合方法．该方法首先分别将高分辨率影像和多光谱影像进行小波分解,然后根据影像的特点,在小波域内进行影像融合,最后通过小波逆变换得到融合结果．通过引入可调节参数,该方法可以在细节保留和光谱保持两方面达到不同程度的平衡,在某些参数组合下,融合图像的目视效果和统计指标可以达到甚至超过传统的小波融合法、IHS变换和主成分变换融合法的效果．     

Remote Sensing Image Fusion Using Multiscale Mapped LS-SVM

The panchromatic (Pan) sharpening of multispectral (MS) bands is an important technique in the various applications of satellite remote sensing. This paper presents an MS Pan- sharpening method using the proposed multiscale mapped least-squares support vector machine (LS-SVM). Under the LS-SVM framework, the salient features underlying the image are represented by support values, and the support value transform (SVT) is developed for image information extraction. The low-resolution MS bands are resampled to the fine scale of the Pan image and sharpened by injecting the detailed features extracted from the high-resolution Pan image. The support value analysis is implemented by using a series of multiscale support value filters that are deduced from the mapped LS-SVM with multiscale Gaussian radial basis function kernels. Experiments are carried out on very high resolution QuickBird MS + Pan data. Fusion simulations on spatially degraded data, whose original MS bands are available for reference, show that the proposed MS Pan-sharpening method performs comparable to the state-of-the-art in terms of the pertained quantitative quality evaluation indexes, such as the Spectral Angle Mapper, relative dimensionless global error in synthesis (ERGAS), modulation-transfer-function-based tool and quality index (Q4), etc. The SVT is an effective tool for remote sensing image fusion.     

Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods

Usual image fusion methods inject features from a high spatial resolution panchromatic sensor into every low spatial resolution multispectral band trying to preserve spectral signatures and improve spatial resolution to that of the panchromatic sensor. The objective is to obtain the image that would be observed by a sensor with the same spectral response (i.e., spectral sensitivity and quantum efficiency) as the multispectral sensors and the spatial resolution of the panchromatic sensor. But in these methods, features from electromagnetic spectrum regions not covered by multispectral sensors are injected into them, and physical spectral responses of the sensors are not considered during this process. This produces some undesirable effects, such as resolution overinjection images and slightly modified spectral signatures in some features. The authors present a technique which takes into account the physical electromagnetic spectrum responses of sensors during the fusion process, which produces images closer to the image obtained by the ideal sensor than those obtained by usual wavelet-based image fusion methods. This technique is used to define a new wavelet-based image fusion method.     

基于采样二通道不可分小波的多光谱图像融合

针对基于非下采样不可分小波图像融合方法空间分辨率不高、基于张量积小波融合方法会出现方块效应的不足,提出了一种基于伸缩矩阵为[1,1;1,-1]的二通道采样不可分小波的多光谱图像和全色图像融合方法.利用矩阵扩充方法,构造了一组新的不可分低通滤波器和高通滤波器组,利用所设计滤波器组分别对多光谱图像的亮度分量和全色图像作下采样的多尺度不可分小波分解,分别对分解后的低频子图像和高频子图像按不同的融合规则进行融合.实验结果表明,其保持光谱信息的能力和保持空间分辨率信息的能力比基于IHS变换融合方法、基于DWT的融合方法、基于IHS-DWT的融合方法、基于IHS-Contourlet变换的融合方法、基于IHS-Curvelet变换的融合方法、SRF方法都强,与基于非下采样的二通道不可分正交小波和不可分双正交小波融合方法相比,该方法能保持较好的整体光谱信息和较高的空间分辨率信息.     

一种基于变分的红外和微光遥感图像融合模型

为了充分利用红外和微光遥感图像中的互补信息,使其便于目视解译,提出了 一种基于变分的图像融合方法。该变分模型定义了细节注入项和结构保真项,在保持红外和微光图像 光谱特性的同时,还改进了融合图像的空间细节和结构特性;引入了正则化能量项,保证了泛函最优解 的平滑性。基于梯度下降流,通过数值迭代获得了融合图像。实验结果表明,该模型能够获取兼具 丰富细节信息和光谱信息的融合图像。与Laplacian金字塔分解方法和多孔小波方法相比,本文 方法具有更佳的融合性能。