IKONOS image fusion process using steepest descent method with bi-linear interpolation

There are many image fusion processes to produce a high-resolution multispectral (MS) image from low-resolution MS and high-resolution panchromatic (PAN) images. But the most significant problems are colour distortion and fusion quality. Previously, we reported a fusion process that produced a1 m resolution IKONOS fused image with minimal spectral distortion. However, block distortion appeared at the edge of the curved sections of the fused image, which was reduced by performing the wavelet transformation as a post-process. Here, we propose an image fusion process using the steepest descent method with bi-linear interpolation, which can remove block distortion without using wavelet transformation. Bi-linear interpolation provides the proper initial values of the fused image, and then the steepest descent method produces the optimum results of the fusion process. These results achieve improvement on the spectral as well as spatial quality of a1 m resolution fused image when compared with other existing methods and remove block distortion completely.     

Multispectral image adaptive pansharpening based on wavelet transformation and NMDB approaches

In remote sensing, satellite images acquired from sensors provide either high spectral or high spatial resolution. The pansharpening framework is applied to remote-sensing systems to enhance the spatial quality of coarse-resolution multispectral (MS) images using information from panchromatic imagery. A multidecomposition pansharpening approach combining MS and panchromatic (PAN) images is proposed in this paper in order to bring the resolution of the low-resolution MS imagery up to that of the panchromatic images. In particular, multilevel wavelet decomposition is applied to the luminance-chrominance (YUV) space transformation (taking into account the red green and blue (RGB) bands) or extended-YUV transformation (taking into account the near infrared (NIR) band in addition to RGB) of the original MS channels, where geometrical details from the panchromatic image are introduced into the MS ones. Our approach contains a preprocessing step that consists of homogenizing the luminance, Y, and the panchromatic image reflectance, which are, respectively, a value integrated over a wavelength spectrum and simply a linear combination of some values in the same spectrum. Hence, as the panchromatic image reflectance and luminance reflectance correspond to different measurements, they do not correspond to the same physical information, which results in a difference between their histograms. Therefore, simple histogram matching is traditionally applied to panchromatic data to fit it to the luminance to avoid colour distortion after fusion. However, as the transformation concerns just the details of the panchromatic and MS images, a new scheme for matching the images which ignores the divergence between their approximations and maximizes the resemblance between their details is proposed in this work. After that, the fusion approach is applied, and in contrast to the original approach where the details of the fused MS luminance are set equal to the PAN luminance, we propose an adaptive approach in which just a part of the PAN details proportional to the similarity between the luminance and lowered PAN image is taken. Indeed, high-resolution geometrical details cannot be similar if the low-resolution details are not in good agreement. Besides, as the agreement between PAN and MS images depends on the occupation class, we have created a segmentation map and then computed separately the correlation in each region. Finally, the evaluation is done based on QuickBird and Pleiades-1A data sets showing rural and suburban areas. When compared to recent methods, our approach provides better results.     

The joint use of IHS transform and redundant wavelet decomposition for fusing multispectral and panchromatic images

Intensity hue saturation (IHS) and wavelet decomposition are two distinct fusion methods used for enhancing the spatial resolution of multispectral images by exploiting a high-resolution panchromatic image. In this paper, a combination of the IHS transform and redundant wavelet decomposition is proposed as a general method for fusing multisensor images. The principle consists of transforming low-resolution multispectral images into IHS independent components. The low-resolution intensity component is fused with the high-resolution panchromatic image in the redundant wavelet domain through an appropriate model. Subsequently, the high-resolution intensity produced is substituted to the low-resolution intensity. High spatial resolution multispectral images are then obtained through an inverse IHS transformation. SPOT images are used to illustrate the superiority of this approach over the IHS fuser in terms of preservation of spectral properties.     

An image fusion method based on object-oriented classification

In image fusion of different spatial resolution multispectral (MS) and panchromatic (PAN) images, a spectrally mixed MS pixel superimposes multiple mixed PAN pixels and multiple pure PAN pixels. This verifies that with increased spatial resolution in imaging, a low spatial resolution spectrally mixed subpixel may be unmixed to be a pure pixel. However, spectral unmixing of mixed MS subpixels is rarely considered in current remote-sensing image fusion methods, resulting in blurred fused images. In the image fusion method proposed in this article, such spectral unmixing is realized. In this method, the MS and PAN images are jointly segmented into image objects, image objects are classified to obtain a classification map of the PAN image and each MS subpixel is fused to be a pixel matching the class of the corresponding PAN pixel. Tested on spatially degraded IKONOS MS and PAN images with a significant spatial resolution ratio of 8:1, the fusion method offered fused images with high spectral quality and deblurred visualization.     

Remote sensing image fusion based on Bayesian linear estimation

A new remote sensing image fusion method based on statistical parameter estimation is proposed in this paper. More specially, Bayesian linear estimation (BLE) is applied to observation models between remote sensing images with different spa- tial and spectral resolutions. The proposed method only estimates the mean vector and covariance matrix of the high-resolution multispectral (MS) images, instead of assuming the joint distribution between the panchromatic (PAN) image and low-resolution multispectral image. Furthermore, the proposed method can enhance the spatial resolution of several principal components of MS images, while the traditional Principal Component Analysis (PCA) method is limited to enhance only the first principal component. Experimental results with real MS images and PAN image of Landsat ETM demonstrate that the proposed method performs better than traditional methods based on statistical parameter estimation, PCA-based method and wavelet-based method.     

Remote sensing image fusion using the curvelet transform

This paper presents an image fusion method suitable for pan-sharpening of multispectral (MS) bands, based on nonseparable multiresolution analysis (MRA). The low-resolution MS bands are resampled to the fine scale of the panchromatic (Pan) image and sharpened by injecting highpass directional details extracted from the high-resolution Pan image by means of the curvelet transform (CT). CT is a nonseparable MRA, whose basis functions are directional edges with progressively increasing resolution. The advantage of CT with respect to conventional separable MRA, either decimated or not, is twofold. Firstly, directional detail coefficients matching image edges may be preliminarily soft-thresholded to achieve a noise reduction that is better than that obtained in the separable wavelet domain. Secondly, modeling of the relationships between high-resolution detail coefficients of the MS bands and of the Pan image is more fitting, being accomplished in the directional multiresolution domain. Experiments are carried out on very-high-resolution MS + Pan images acquired by the QuickBird and Ikonos satellite systems. Fusion simulations on spatially degraded data, whose original MS bands are available for reference, show that the proposed curvelet-based fusion method performs slightly better than the state-of-the art. Fusion tests at the full scale reveal that an accurate and reliable Pan-sharpening, little affected by local inaccuracies even in the presence of complex and detailed urban landscapes, is achieved by the proposed method.     

PAN‐sharpening of very high resolution multispectral images using genetic algorithms

A novel image fusion method is presented, suitable for sharpening of multispectral (MS) images by means of a panchromatic (PAN) observation. The method is based on redundant multiresolution analysis (MRA); the MS bands expanded to the finer scale of the PAN band are sharpened by adding the spatial details from the MRA representation of the PAN data. As a direct, unconditioned injection of PAN details gives unsatisfactory results, a new injection model is proposed that provides the optimum injection by maximizing a global quality index of the fused product. To this aim, a real‐valued genetic algorithm (GA) has been defined and tested on Quickbird data. The optimum GA injection is driven by an index function capable of measuring different types of possible distortions in the fused images. Fusion tests are carried out on spatially degraded data to objectively compare the proposed scheme to the most promising state‐of‐the‐art image fusion methods, and on full‐resolution image data to visually assess the performance of the proposed genetic image fusion method.     

A genetic algorithm solution to the gram-schmidt image fusion

ABSTRACT

There is no such thing as ‘the best image fusion method’ in terms of both spectral and spatial fidelity. This fact encourages the researchers to develop more advanced approaches in order to optimally transfer the spatial details without distorting the colour content. Component substitution (CS)-based image fusion methods have been proven to produce sharper images but suffer from colour distortion. The aim of this study was to modify the CS-based Gram-Schmidt (GS) fusion method with the aid of the Genetic Algorithm (GA) to further improve its colour preservation performance. The GA was used to estimate a weight for each multispectral (MS) band. The obtained band weights were used to generate a low-resolution panchromatic (PAN) band, which plays a significant role in the performance of the GS method. The performance of the proposed approach was compared not only against the conventional GS, but also against widely-used CS-based, multiresolution analysis (MRA)-based and colour-based (CB) image fusion methods. The results indicated that the proposed GA-based approach produced spectrally and spatially superior results compared to the other methods used.     

Image misalignment caused by decimation in image fusion evaluation

In the evaluation of image fusion methods, spatially degraded multispectral (MS) and panchromatic (PAN) images are frequently employed as test data sets. The degradation is implemented using either averaging or a combination of low-pass filtering and decimation. However, the decimation operation causes the degraded MS and PAN images to be slightly misaligned with each other and with the original MS image, which acts as the reference image in the fusion evaluation. In this study, two image fusion methods based on decimated and undecimated multiresolution analysis techniques were evaluated on three popular types of test data sets consisting of spatially degraded IKONOS MS and PAN images. In the experiment, image misalignments caused by decimation significantly influenced the quality of fused images and resulted in untrustworthy performances of the image fusion methods being evaluated. It was demonstrated that unlike aligned MS and PAN images in actual image fusion, misaligned MS and PAN images in test data sets are inappropriate for image fusion evaluation.     

改进空间细节提取策略的分量替换遥感图像融合方法

针对多光谱图像与全色图像间的局部空间差异引起的空谱失真问题,提出了一种改进空间细节提取策略的分量替换遥感图像融合方法。与传统空间细节提取方法不同,该方法旨在合成高质量的强度图像,用其取代空间细节提取步骤中全色图像的位置,以获取匹配多光谱图像的空间细节信息。首先,借助低分辨率强度图像与高分辨率强度图像的流形结构一致性,利用基于局部线性嵌入的图像重建方法重构第一幅高分辨率强度图像;其次,对低分辨率强度图像与全色图像分别进行小波分解,保留低分辨率强度图像的低频信息与全色图像的高频信息,利用逆小波变换重构第二幅高分辨率强度图像;然后,将两幅高分辨率强度图像进行稀疏融合,获得高质量强度图像;最后,将合成的高分辨率强度图像应用到分量替换融合框架,获取最终融合图像。实验结果表明,与另外11种融合方法相比,所提方法得到的融合图像具有较高的空间分辨率和较低的光谱失真度,该方法的平均相关系数、均方根误差、相对整体维数合成误差、光谱角匹配指数和基于四元数理论的指标在三组GeoEye-1融合图像上的均值分别为：0.9439、24.3479、2.7643、3.9376和0.9082,明显优于对比方法的相应评价指标。该方法可有效地消除局部空间差异对分量替换融合框架性能的影响。     

A Variational Model for P+XS Image Fusion

We propose an algorithm to increase the resolution of multispectral satellite images knowing the panchromatic image at high resolution and the spectral channels at lower resolution. Our algorithm is based on the assumption that, to a large extent, the geometry of the spectral channels is contained in the topographic map of its panchromatic image. This assumption, together with the relation of the panchromatic image to the spectral channels, and the expression of the low-resolution pixel in terms of the high-resolution pixels given by some convolution kernel followed by subsampling, constitute the elements for constructing an energy functional (with several variants) whose minima will give the reconstructed spectral images at higher resolution. We discuss the validity of the above approach and describe our numerical procedure. Finally, some experiments on a set of multispectral satellite images are displayed.     

Pan-sharpening using a guided filter

Pan-sharpening aims to integrate the spatial details of a high-resolution panchromatic (Pan) image with the spectral information of low-resolution multispectral (MS) images to produce high-resolution MS images. The key is to appropriately estimate the missing spatial details of the MS images while preserving their spectral contents. However, many existing methods extract the spatial details from the Pan image without fully considering the structures of the MS images, resulting in spectral distortion due to redundant detail injection. A guided filter can transfer the structures of the MS images into the intensity component or the low-pass approximation of the Pan image. Using the guided filter, we propose two novel pan-sharpening methods to reduce the redundant details among the MS and Pan images. Specifically, we extract the missing spatial details of the MS images by minimizing the difference between the Pan image and its corresponding filtering output, with the help of the MS images. Two different ways of using the MS images as guided images lead to two proposed methods, which can be grouped into component substitution (CS) family. Extensive experimental results over three data sets collected by different satellite sensors demonstrate the effectiveness of the proposed methods.     

Pansharpening through an improved sparsity-based fusion method

ABSTRACT

In this paper, a patch-wise manner based on the sparsity is proposed to fuse a panchromatic (PAN) image and a low resolution multispectral (LMS) image. In the sparsity-based pansharpening methods, improving the training process of the dictionaries and sparse coefficients of the fused image, which is the main goal of this paper, have a significant impact on the fused results. In this paper, the fused image is obtained by minimizing the cost function which is obtained from incorporating a Markov random field (MRF)-based prior model into the maximum a posteriori (MAP) estimation. The contribution of this paper is twofold derived from our proposed prior model. 1) The prior model only involves the parts of the PAN information related to a considered band of the high-resolution multispectral image in the training process of the dictionary of the considered band. Not only does it improve the training of the dictionaries, but it also leads to finding more accurate sparse coefficients. 2) The high-frequency information of the PAN image is also involved in the training process as a separate term. This term decreases the spectral distortion by relieving the adverse effect of dissimilarity of the grey levels between the PAN and multispectral images on the fused image. The visual and quantitative comparison between the performance of the proposed method and eight well-known fusion methods on the Pleiades, QuickBird, and DEIMOS-2 data demonstrate the superiority of the proposed method.     

Two improvement schemes of PAN modulation fusion methods for spectral distortion minimization

Fusion of panchromatic (PAN) and multispectral (MS) images is one of the most promising issues in remote sensing. PAN modulation fusion methods are usually based on an assumption that a ratio of two different‐resolution versions of an MS band is equal to a ratio of two different‐resolution versions of a PAN image. In such fusion methods, image haze is rarely taken into account, and it may produce serious spectral distortion in synthetic images. In this paper, assuming that the previous ratio relationship only holds for haze‐free images, two relevant improvement schemes are proposed to better express the ratio relationship of haze‐included images. In a test on a spatially degraded IKONOS dataset, the first scheme synthesizes an image with minimum spectral distortion, and the second modifies several current PAN modulation fusion methods and generates high‐quality synthetic products. The experiment results confirm that image haze can seriously impact the quality of fused images obtained by using PAN modulation fusion methods, and it should be taken into account in relevant image fusion.     

Fusion of multispectral and panchromatic images based on support value transform and adaptive principal component analysis

In this paper we combined the projection-substitution with ARSIS (French acronym for “Amélioration de la Résolution Spatiale par Injection de Structures”, i.e., Improving Spatial Resolution by Structure Injection) concept assumption for fusion of panchromatic (PAN) and multispectral (MS) images. Firstly support value filter (SVF) is used to establish a new multiscale model (MSM), support vector transform (SVT), and adaptive principal component analysis (APCA) is then employed to select the principal components of MS images by means of a statistical measure of the correlation between MS and PAN images; secondly, a local approach is used to check whether a structure should appear in the new principal component and PAN high frequency structures are transformed by high resolution interband structure model (HRIBSM) before inserting in the MS modalities. Because SVT is an undecimated, dyadic and aliasing transform with shift-invariant property, the fused image can avoid ringing effects suffered from sampling. Additionally, the ARSIS concept can make full use of the remote sensing physics to reduce the spatial and spectrum distortion in the structure injection. Texture extraction is also employed to avoid the spectral distortion caused by the mistaken injection of low-pass components into the MS images. Experimental results including visual and numerical evaluation also proves the superiority of the proposed method to its counterparts.     

Iterative scheme for MS image pansharpening based on the combination of multi-resolution decompositions

Image pansharpening in the remote-sensing domain may be defined as the technique of extracting high-resolution details from the panchromatic (PAN) image and injecting them into the multispectral (MS) one in a way to preserve the spectral signature and improve the spatial resolution. In this article, the authors propose an image fusion framework that tries to derive sharpened MS image such that: (i) when decimated taking into account the imagery system Modulation Transfer Function (MTF), it equals the original MS image; (ii) when decomposed using discrete wavelet transform (DWT), its geometrical details are those of the PAN image weighted by the compatibility PAN/MS. Indeed, MS sharpening is carried out in two steps. First, pre-pansharpened MS image is obtained using inverse DWT taking as approximations those of the upsampled original MS image and as details those of PAN (to reduce spectral distortion, PAN detail injection is performed proportionally to the similarity PAN/MS). Second, to satisfy (i) and to remove the PAN-MS disagreement, an iteration algorithm (alternatively corrects approximations and details) has been proposed. The proposed approach is designed in two versions inspired by the Generalized Laplacian Pyramid (GLP) and the Gram–Schmidt (GS) transformation, respectively.

To validate our approach, Pléiades-1A, Geoeye-1, and Landsat Enhanced Thematic Mapper Plus (ETM+) images are tested. The results of qualitative and quantitative scores are presented and discussed. Compared to well-known techniques, our approach shows generally better results, particularly the one based on GLP formalism.     

基于双模式联合三边滤波器的深度图像超分辨率方法

彩色图像引导的深度图像超分辨率方法通过利用高分辨率彩色图像的高频信息来重建深度图像,取得了不错的重建效果,但当深度图像和彩色图像边缘不(完全)一致或彩色区域纹理丰富时,重建图像普遍存在边缘模糊和纹理拷贝问题.针对这一问题,提出一种边缘图像引导的双模式联合三边滤波器(DMJTF)方法.该方法利用单幅低分辨率深度图像构建了一个边缘图像金字塔字典,然后利用MRF模型构建了一个高分辨率边缘图像,该图像确定了滤波器的两种模式,分别用于重构深度图像的边缘和平滑区域.实验结果表明DMJTF算法有效地避免了纹理拷贝异常,降低了边缘模糊现象,在定性和定量两个方面都优于其他算法,取得了较好的超分效果.     

形态学滤波和改进PCNN的NSST域多光谱与全色图像融合

目的 全色图像的空间细节信息增强和多光谱图像的光谱信息保持通常是相互矛盾的,如何能够在这对矛盾中实现最佳融合效果一直以来都是遥感图像融合领域的研究热点与难点。为了有效结合光谱信息与空间细节信息,进一步改善多光谱与全色图像的融合质量,提出一种形态学滤波和改进脉冲耦合神经网络（PCNN）的非下采样剪切波变换（NSST）域多光谱与全色图像融合方法。方法 该方法首先分别对多光谱和全色图像进行非下采样剪切波变换;对二者的低频分量采用形态学滤波和高通调制框架（HPM）进行融合,将全色图像低频子带的细节信息注入到多光谱图像低频子带中得到融合后的低频子带;对二者的高频分量则采用改进脉冲耦合神经网络的方法进行融合,进一步增强融合图像中的空间细节信息;最后通过NSST逆变换得到融合图像。结果 仿真实验表明,本文方法得到的融合图像细节信息清晰且光谱保真度高,视觉效果上优势明显,且各项评价指标与其他方法相比整体上较优。相比于5种方法中3组融合结果各指标平均值中的最优值,清晰度和空间频率分别比NSCT-PCNN方法提高0.5%和1.0%,光谱扭曲度比NSST-PCNN方法降低4.2%,相关系数比NSST-PCNN方法提高1.4%,信息熵仅比NSST-PCNN方法低0.08%。相关系数和光谱扭曲度两项指标的评价结果表明本文方法相比于其他5种方法能够更好地保持光谱信息,清晰度和空间频率两项指标的评价结果则展示了本文方法具有优于其他对比方法的空间细节注入能力,信息熵指标虽不是最优值,但与最优值非常接近。结论 分析视觉效果及各项客观评价指标可以看出,本文方法在提高融合图像空间分辨率的同时,很好地保持了光谱信息。综合来看,本文方法在主观与客观方面均具有优于亮度色调饱和度（IHS）法、主成分分析（PCA）法、基于非负矩阵分解（CNMF）、基于非下采样轮廓波变换和脉冲耦合神经网络（NSCT-PCNN）以及基于非下采样剪切波变换和脉冲耦合神经网络（NSST-PCNN）5种经典及现有流行方法的融合效果。     

高分辨率遥感影像融合存在的问题及改进措施

影像融合技术作为富集和优化多源遥感影像信息的一种有效途径一直受到遥感界和图像处理领域的关注，并且已开发出多种融合方法。但是随着高分辨率遥感卫星的发展。当前一些融合方法对新型高分辨率遥感影像如IKONOS、Quick—Bird的融合效果欠佳，颜色差异是存在的主要问题。本文从高分辨率遥感影像光谱特征的分析入手找到颜色偏差的原因，同时给出了多种改进措施．然后分别利用传统的和改进的方法进行了融合试验，并对试验结果进行对比分析，以期找到对新型高分辨率遥感影像融合更加适用的方法。     

基于流形学习的彩色遥感图像分维数估算

目的 纹理特征提取一直是遥感图像分析领域研究的热点和难点。现有的纹理特征提取方法主要集中于研究单波段灰色遥感图像,如何提取多波段彩色遥感图像的纹理特征,是多光谱遥感的研究前沿。方法 提出了一种基于流形学习的彩色遥感图像分维数估算方法。该方法利用局部线性嵌入方法,对由颜色属性所组成的5-D欧氏超曲面进行维数简约处理;再将维数简约处理后的颜色属性用于分维数估算。结果 利用Landsat-7遥感卫星数据和GeoEye-1遥感卫星数据进行实验,结果表明,同Peleg法和Sarkar法等其他分维数估算方法相比,本文方法具有较小的拟合误差。其中,其他4种对比方法所获拟合误差E平均值分别是本文方法所获得拟合误差E平均值的26.2倍、5倍、26.3倍、5倍。此外,本文方法不仅可提供具有较好分类特性的分维数,而且还能提供相对于其他4种对比方法更加稳健的分维数。结论 在针对中低分辨率的真彩遥感图像和假彩遥感图像以及高分辨率彩色合成遥感图像方面,本文方法能够利用不同地物所具有颜色属性信息,提取出各类型地物所对应的纹理信息,有效地改善了分维数对不同地物的区分能力。这对后续研究各区域中不同类型地物的分布情况及针对不同类型地物分布特点而制定区域规划及开发具有积极意义。