Sentinel-2 Satellite Image Fusion Methodand Quality Evaluation Analysis

Sentinel-2 satellite sensors acquire three kinds of optical remote sensing images with different spatial resolutions.How to improve the spatial resolution of lower spatial resolution bands by fusion method is one of the problems faced by Sentinel-2 applications.Taking the Sentinel\|2B image as the data source,a high spatial resolution band was generated or selected from the four 10m spatial resolution bands by four methods:the maximum correlation coefficient,the central wavelength nearest neighbor,the pixel maximum and the principal component analysis.We fused the one high spatial resolution band produced and six multispectral bands with 20 m spatial resolution by the five fusion methods of PCA,HPF,WT,GS and Pansharp to produce six multispectral bands with 10 m spatial resolution and the fusion results were evaluated from three aspects:qualitative and quantitative (information entropy,average gradient,spectral correlation coefficient,root mean square error and general image quality index) and classification accuracy of fused images.Results show that the fusion quality of Pansharp with the maximum correlation coefficient is better than other fusion methods,and the classification accuracy is slightly lower than the GS with the pixel maximum of the highest classification accuracy and far higher than the original four multispectral image with 10 m spatial resolution.According to the classification accuracy of experimental data,different fusion methods have different advantages in extraction of different ground objects.In application,appropriate schemes should be selected according to actual research needs.This research can provide reference for Sentinel-2 satellite and similar satellite data processing and application.     

异源、同源传感器影像融合的比较研究

遥感影像的融合是遥感界的一个研究热点。根据数据源的不同,影像融合可分为异源传感器影像融合和同源传感器影像融合。以TM与SPOT作为异源影像融合的例子,以IKONOS的MS与Pan作为同源影像融合的例子,用5种算法对两种融合类型进行实验与比较。结果表明,同源传感器影像的融合效果好于异源传感器影像的融合效果;不同的融合算法在异源和同源传感器影像融合中的表现不尽相同。SVR变换可同时应用于异源及同源传感器影像的融合,且在提高影像空间分辨率、信息量和清晰度的同时能很好地保持原始多光谱影像的光谱特征。SFIM虽然也可以在两种数据源的融合实验中获得较好的融合效果,但其高频信息融入度最差。MB虽然提高了融合影像的高频信息融入程度,但光谱保真度、信息量和清晰度却不理想。Ehlers适用于异源传感器影像间的融合,而WT则适用于同源传感器影像的融合。     

Multispectral images fusion by a joint multidirectional and multiresolution representation

This paper proposes a new fusion method that permits an adequate selection of information extracted from source images to obtain fused images with good spatial and spectral quality simultaneously. This method is based on a joint multiresolution multidirectional representation of the source images using a single directional spatial frequency low pass filter bank of low computational complexity, defined in the Fourier domain. The source images correspond to those captured by the IKONOS satellite (panchromatic and multispectral). The results obtained indicate that the proposed method provides, in a simple manner, objective control over the trade‐off between high spatial and spectral quality of the fused images.     

基于归一化相关矩的多分辨率遥感图象融合

多传感器数据融合技术已广泛应用于遥感图象处理方面 .针对遥感多光谱图象空间分辨率较低的问题 ,提出了一种基于归一化相关矩的多分辨率图象融合方法 .该方法首先对图象进行二维小波变换 ,然后根据所得到的高频小波系数的一阶、二阶统计特征来定义图象局部灰度相关矩 ,并以此作为图象融合测度来对遥感图象进行多分辨率特征融合 ,从而得到包含更多信息和有效特征的融合图象 .仿真结果表明 ,融合后的图象在保留多光谱信息和提高空间分辨率上均能获得较好的效果 ,因而可以更好地用于目标识别、分类等遥感图象处理方面     

基于改进NSCT和IHS变换的遥感影像融合方法

目的 为了增强多光谱和全色影像融合质量,提出基于脉冲耦合神经网络（PCNN）的非下采样Contoulet变换（NSCT）和IHS变换相结合的融合方法。方法 先对多光谱图像进行IHS变换提取亮度I分量,采用主成分分析增强I分量得到新的I+分量;然后通过NSCT变换分别对I+分量和全色图像进行分解,并采用边缘梯度信息激励的PCNN得到融合图像的低频和高频分量;最后进行NSCT逆变换、IHS逆变换得到融合图像。结果 利用资源一号02C卫星数据进行实验,结果表明该算法在保留光谱信息的同时提高了图像空间分辨率,获得了较好的融合效果。结论 结合NSCT和IHS变换的融合方法在视觉效果和客观评价指标上都优于常用的图像融合方法。     

Fusion of multispectral and panchromatic satellite sensor imagery based on tailored filtering in the Fourier domain

A new methodology for fusing satellite sensor imagery, based on tailored filtering in the Fourier domain is proposed. Finite‐duration Impulse Response (FIR) filters have been designed through an objective criterion, which depends on source image characteristics only. The designed filters allow a weighted fusion of the information contained in a fine spatial resolution image (PAN) and in a multispectral image (MULTI), respectively, establishing a trade‐off between spatial and spectral quality of the resulting fused image. This new technique has been tested with Landsat Enhanced Thematic Mapper Plus (ETM+) imagery. Spatial and spectral quality of the fused images was compared with the results provided by Mallat's Wavelet algorithm. The images fused by the proposed method were characterized by a spatial resolution very close to the PAN image, and by the spectral resolution of the MULTI image.     

Image Fusion Based on NSCT and Sparse Representation for Remote Sensing Data

The practice of integrating images from two or more sensors collected from the same area or object is known as image fusion. The goal is to extract more spatial and spectral information from the resulting fused image than from the component images. The images must be fused to improve the spatial and spectral quality of both panchromatic and multispectral images. This study provides a novel picture fusion technique that employs L0 smoothening Filter, Non-subsampled Contour let Transform (NSCT) and Sparse Representation (SR) followed by the Max absolute rule (MAR). The fusion approach is as follows: first, the multispectral and panchromatic images are divided into lower and higher frequency components using the L0 smoothing filter. Then comes the fusion process, which uses an approach that combines NSCT and SR to fuse low frequency components. Similarly, the Max-absolute fusion rule is used to merge high frequency components. Finally, the final image is obtained through the disintegration of fused low and high frequency data. In terms of correlation coefficient, Entropy, spatial frequency, and fusion mutual information, our method outperforms other methods in terms of image quality enhancement and visual evaluation.     

Remote-sensing image fusion using sparse representation with sub-dictionaries

Remote-sensing image fusion aims to obtain a multispectral (MS) image with a high spatial resolution, which integrates spatial information from the panchromatic (Pan) image and with spectral information from the MS image. Sparse representation (SR) has been recently used in remote-sensing image fusion method, and can obtain superior results to many traditional methods. However, the main obstacle is that the dictionary is generated from high resolution MS images (HRMS), which are difficult to acquire. In this article, a new SR-based remote-sensing image fusion method with sub-dictionaries is proposed. The image fusion problem is transformed into a restoration problem under the observation model with the sparsity constraint, so the fused HRMS image can then be reconstructed by a trained dictionary. The proposed dictionary for image fusion is composed of several sub-dictionaries, each of which is constructed from a source Pan image and its corresponding MS images. Therefore, the dictionary can be constructed without other HRMS images. The fusion results from QuickBird and IKONOS remote-sensing images demonstrate that the proposed method gives higher spatial resolution and less spectral distortion compared with other widely used and the state-of-the-art remote-sensing image fusion methods.     

A novel adaptive fast IHS transform fusion method driven by regional spectral characteristics for Gaofen-2 imagery

ABSTRACT

With the increasing diversity of applications based on the Gaofen-2 satellite imagery, broadly applicable methods to generate high quality fused images is a significant problem to investigate. To obtain an image with high spatial and spectral resolutions from given panchromatic (Pan) images and multispectral (MS) images, most existing fusion algorithms adopt a unified strategy for the whole image. However, regions have distinct characteristics that impact the spatial and spectral resolution processing, on account of their varying regional features. In this article, to satisfy the diverse needs of different regions, a novel fast IHS (Intensity-Hue-Saturation) transform fusion method driven by regional spectral characteristics is proposed to fuse Gaofen-2 imagery. First, by the fast IHS transform framework, the original intensity component is obtained from the upsampled MS imagery. Then, numerous independent regions of upsampled MS imagery are generated by a novel superpixel merging strategy, and the spectral characteristics of these regions are utilized for generating a fusion factor. Next, to acquire a new fused intensity component, the fusion factor is applied to guide the injection of details in the fusion procedure. This fusion factor adapts the method to meet the spatial and spectral resolution needs for each region. Finally, the difference between the new fused intensity component and the original one is regarded as the detail that needs to be injected; these are added equally to the different bands of the upsampled MS imagery to yield the final fused multispectral image. In comparison with other classical algorithms, the visual and statistical analysis reveal that our proposed method can provide better results in improving spatial detail and preserving spectral information.     

多光谱图像与全色图像的像素级融合研究

以高空间分辨率的全色图像与高光谱分辨率的多光谱图像进行像素级双源融合为例，详细地总结了卫星多源遥感图像融合领域像素级融合的步骤、基本融合模型和优缺点，重点分析了各种常见像素级融合方法的原理和特点，并归纳了像素级融合结果的主客观评价标准和评价方法，以及像素级融合的主要应用领域，最后讨论了像素级融合目前存在的问题和今后的发展方向。     

基于Iαβ色彩空间和Contourlet变换相结合的融合方法*

针对北京1号小卫星的多光谱与全色波段的分辨率比率较大,传统的融合方法会产生边界模糊和光谱扭曲现象,提出了一种新的融合算法。首先对多光谱与全色影像分别进行Iαβ和Contourlet变换;然后在频率域中采用不同的融合策略进行处理;最后进行Contourlet和Iαβ逆变换,得到融合图像。实验表明,本方法既提高了融合图像的空间细节信息又很好地保持了图像的光谱特征,优于传统的融合方法。     

An image fusion method based on object-oriented classification

In image fusion of different spatial resolution multispectral (MS) and panchromatic (PAN) images, a spectrally mixed MS pixel superimposes multiple mixed PAN pixels and multiple pure PAN pixels. This verifies that with increased spatial resolution in imaging, a low spatial resolution spectrally mixed subpixel may be unmixed to be a pure pixel. However, spectral unmixing of mixed MS subpixels is rarely considered in current remote-sensing image fusion methods, resulting in blurred fused images. In the image fusion method proposed in this article, such spectral unmixing is realized. In this method, the MS and PAN images are jointly segmented into image objects, image objects are classified to obtain a classification map of the PAN image and each MS subpixel is fused to be a pixel matching the class of the corresponding PAN pixel. Tested on spatially degraded IKONOS MS and PAN images with a significant spatial resolution ratio of 8:1, the fusion method offered fused images with high spectral quality and deblurred visualization.     

改进NSCT和IHS变换相结合的遥感影像融合

目的 目的为了增强多光谱和全色影像融合质量,提出基于脉冲耦合神经网络（PCNN）的非下采样Contoulet变换（NSCT）和IHS变换相结合的融合方法。方法 先对多光谱图像进行IHS变换提取亮度I分量,采用主成分分析增强I分量得到新的I⁺分量;然后通过NSCT变换分别对I⁺分量和全色图像进行分解,并采用边缘梯度信息激励的PCNN得到融合图像的低频和高频分量;最后进行NSCT逆变换、IHS逆变换得到融合图像。结果 利用资源一号02C卫星数据进行实验,结果表明该算法在保留光谱信息的同时提高了图像空间分辨率,获得了较好的融合效果。结论 结合NSCT和IHS变换的融合方法在视觉效果和客观评价指标上都优于常用的图像融合方法。     

Fusion and registration of THEOS multispectral and panchromatic images

This article presents a new method for the fusion and registration of THEOS (Thailand Earth Observation Satellite) multispectral and panchromatic images in a single step. In the usual procedure, fusion is an independent process separated from the registration process. However, both image registration and fusion can be formulated as estimation problems. Hence, the registration parameters can be automatically tuned so that both fusion and registration can be optimized simultaneously. Here, we concentrate on the relationship between low-resolution multispectral and high-resolution panchromatic imagery. The proposed technique is based on a statistical framework. It employs the maximum a posteriori (MAP) criterion to jointly solve the fusion and registration problem. Here, the MAP criterion selects the most likely fine resolution multispectral and mapping parameter based on observed coarse resolution multispectral and fine resolution panchromatic images. The Metropolis algorithm was employed as the optimization algorithm to jointly determine the optimum fine resolution multispectral image and mapping parameters. In this work, a closed-form solution that can find the fused multispectral image with correcting registration is also derived. In our experiment, a THEOS multispectral image with high spectral resolution and a THEOS panchromatic image with high spatial resolution are combined to produce a multispectral image with high spectral and spatial resolution. The results of our experiment show that the quality of fused images derived directly from misaligned image pairs without registration error correction can be very poor (blurred and containing few sharp edges). However, with the ability to jointly fuse and register an image pair, the quality of the resulting fused images derived from our proposed algorithm is significantly improved, and, in the simulated cases, the fused images are very similar to the original high resolution multispectral images, regardless of the initial registration errors.     

Effects of spatial and spectral resolution in estimating ecosystem α-diversity by satellite imagery

Remote sensing represents a powerful tool to derive quantitative and qualitative information about ecosystem biodiversity. In particular, since plant species richness is a fundamental indicator of biodiversity at the community and regional scales, attempts were made to predict species richness (spatial heterogeneity) by means of spectral heterogeneity. The possibility of using spectral variance of satellite images for predicting species richness is known as Spectral Variation Hypothesis. However, when using remotely sensed data, researchers are limited to specific scales of investigation. This paper aims to investigate the effects of scale (both as spatial and spectral resolution) when searching for a relation between spectral and spatial (related to plant species richness) heterogeneity, by using satellite data with different spatial and spectral resolution. Species composition was sampled within square plots of 100 m² nested in macroplots of 10,000 m². Spectral heterogeneity of each macroplot was calculated using satellite images with different spatial and spectral resolution: a Quickbird multispectral image (4 bands, spatial resolution of 3 m), an Aster multispectral image (first 9 bands used, spatial resolution of 15 m for bands 1 to 3 and 30 m for bands 4 to 9), an ortho-Landsat ETM+ multispectral image (bands 1 to 5 and band 7 used; spatial resolution, 30 m), a resampled 60 m Landsat ETM+ image.Quickbird image heterogeneity showed a statistically highly significant correlation with species richness (r = 0.69) while coarse resolution images showed contrasting results (r = 0.43, r = 0.67, and r = 0.69 considering the Aster, Landsat ETM+, and the resampled 60 m Landsat ETM+ images respectively). It should be stressed that spectral variability is scene and sensor dependent. Considering coarser spatial resolution images, in such a case even using SWIR Aster bands (i.e. the additional spectral information with respect to Quickbird image) such an image showed a very low power in catching spectral and thus spatial variability with respect to Landsat ETM+ imagery. Obviously coarser resolution data tend to have mixed pixel problems and hence less sensitive to spatial complexity. Thus, one might argue that using a finer pixel dimension should inevitably result in a higher level of detail. On the other hand, the spectral response from different land-cover features (and thus different species) in images with higher spectral resolution should exhibit higher complexity.Spectral Variation Hypothesis could be a basis for improving sampling designs and strategies for species inventory fieldwork. However, researchers must be aware on scale effects when measuring spectral (and thus spatial) heterogeneity and relating it to field data, hence considering the concept of scale not only related to a spatial framework but even to a spectral one.     

Spatially adaptive multi-resolution multispectral image fusion

Due to the performance limit of remote sensing systems, multispectral images have limited spatial resolution. Their spatial resolution can be improved by merging them with higher resolution image data. A fundamental problem frequently occurring in existing fusion processes, however, is the distortion of spectral information. This paper presents a spatially adaptive image fusion algorithm which produces visually natural images and retains the quality of local spectral information as well. High frequency information of the high resolution image to be inserted to the resampled multispectral images is controlled by adaptive gains to incorporate the difference of local spectral characteristics between the high and the low resolution images into the fusion. Each gain is estimated to minimize the l ₂-norm of the error between the original and the estimated pixel values defined in a spatially adaptive window of which the weights are proportional to the spectral correlation measurements of the corresponding regions. This method is applied to a set of co-registered Landsat 7 Enhanced Thematic Mapper (ETM)+ panchromatic and multispectral image data. The experimental results show that high resolution images can be synthesized by the proposed method, which successfully preserves spectral content of the multispectral images.     

改进空间细节提取策略的分量替换遥感图像融合方法

针对多光谱图像与全色图像间的局部空间差异引起的空谱失真问题,提出了一种改进空间细节提取策略的分量替换遥感图像融合方法。与传统空间细节提取方法不同,该方法旨在合成高质量的强度图像,用其取代空间细节提取步骤中全色图像的位置,以获取匹配多光谱图像的空间细节信息。首先,借助低分辨率强度图像与高分辨率强度图像的流形结构一致性,利用基于局部线性嵌入的图像重建方法重构第一幅高分辨率强度图像;其次,对低分辨率强度图像与全色图像分别进行小波分解,保留低分辨率强度图像的低频信息与全色图像的高频信息,利用逆小波变换重构第二幅高分辨率强度图像;然后,将两幅高分辨率强度图像进行稀疏融合,获得高质量强度图像;最后,将合成的高分辨率强度图像应用到分量替换融合框架,获取最终融合图像。实验结果表明,与另外11种融合方法相比,所提方法得到的融合图像具有较高的空间分辨率和较低的光谱失真度,该方法的平均相关系数、均方根误差、相对整体维数合成误差、光谱角匹配指数和基于四元数理论的指标在三组GeoEye-1融合图像上的均值分别为：0.9439、24.3479、2.7643、3.9376和0.9082,明显优于对比方法的相应评价指标。该方法可有效地消除局部空间差异对分量替换融合框架性能的影响。     

基于l αβ空间的多光谱和全色图像融合

提出了一种新的基于lαβ空间的图像融合方法，该方法可以用来对低分辨率的多光谱图像和高分辨率的全色图像进行融合。该方法通过对多光谱图像和全色图像的融合，得到一幅融合后的图像，该融合后的图像集合了多光谱图像的光谱信息和全色图像的空间信息。实验结果表明该方法效果良好，优于传统的以及改进的IHS方法和PCA方法。     

Comparison of different pan-sharpening methods for spectral characteristic preservation: multi-temporal CARTOSAT-1 and IRS-P6 LISS-IV imagery

A host of remote-sensing and mapping applications require both high spatial and high spectral resolutions. Availability of high spatial and spectral details at different resolutions from a suite of satellite sensors has necessitated the development of effective image fusion techniques that can effectively combine the information available from different sensors and take advantage of their varied capabilities. A common problem observed in the case of multi-sensor multi-temporal data fusion is spectral distortion of the fused images. Performance of a technique also varies with variation in scene characteristics. In this article, two sets of multi-temporal CARTOSAT-1 and Indian Remote Sensing satellite (IRS-P6) Linear Imaging and Self Scanning sensor (LISS-IV) image sub-scenes, with different urban landscape characteristics, are fused with an aim to evaluate the performance of five image fusion algorithms – high-pass filtering (HPF), Gram–Schmidt (GS), Ehlers, PANSHARP and colour-normalized Brovey (CN-Brovey). The resultant fused data sets are compared qualitatively and quantitatively with respect to spectral fidelity. Spatial enhancement is assessed visually. The difference in the performance of techniques with variation in scene characteristics is also examined. For both scenes, GS, HPF and PANSHARP fusion techniques produced comparable results with high spectral quality and spatial enhancement. For these three methods, the variation in performance over different scenes was not very significant. The Ehlers method resulted in spatially degraded images with a more or less constant negative offset in data values in all bands of one scene and in the first two bands in the other. The CN-Brovey method produced excellent spatial enhancement but highly distorted radiometry for both sub-scenes.     

基于PCA和NSCT的多光谱图像和全色图像的融合

研究了主分量分析（PCA）和非下采样Contourlet变换（NSCT）,提出一种新的多光谱图像和全色图像的融合算法。该方法对多光谱图像进行PCA变换,对所得的第一主分量（PC1）以及全色图像进行NSCT变换。对二者的低频近似系数再次进行PCA变换以寻求多光谱信息和空间信息的平衡;对于高频细节系数,通过结构相似性指标（SSIM）和局部Sobel梯度进行融合,进一步提高空间信息量;经过逆NSCT和逆PCA变换得到融合图像。实验结果表明,提出的方法在增强融合图像空间细节表现能力的同时,尽可能地保留了多光谱图像的光谱信息,优于传统的基于IHS、PCA、小波变换和Contourlet变换的融合方法,是有效可行的。