Assessment of pan-sharpened very high-resolution WorldView-2 images

The fusion of multispectral (MS) images with high spatial resolution panchromatic (pan) images compensates the trade-off between spatial and spectral resolutions. The performance of fusion algorithms for different sensors is an active area of research. Therefore, with the availability of new very high-resolution (VHR) sensors, it becomes customary to evaluate the applicability of existing fusion techniques. In this study, we focused on the WorldView-2 (WV2) spaceborne sensor, which captures data in eight MS (2 m spatial resolution) bands and one pan (0.5 m spatial resolution) band. We compared and assessed 12 fusion techniques, namely Brovey transform (BT), Ehlers, Gram–Schimdt (GS), hyperspherical colour sphere (HCS), high-pass filter (HPF), modified intensity hue saturation (ModIHS), Multiplicative, PANSHARP, PANSHARP2, principal component (PC), and wavelet (IHS and PC)-based methods. To measure the quality of fused products, qualitative and quantitative methods were used. In qualitative methods, visual analysis of different colour composites was carried out. In quantitative methods, band-wise eight quality metrics including the required processing time and controlling parameters were reported. For overall image quality assessment, it is necessary to combine the values of different quality metrics. Therefore, a mean observation score based on these values was calculated to rank the fusion methods. It was observed that the HPF and PANSHARP methods produced the most visually appealing images, whereas quantitative values indicated that HCS, HPF, and PANSHARP methods performed better than other methods. Combining the results of quantitative and qualitative analyses, it was found that PANSHARP and HPF methods are superior to other methods in preserving both spatial and spectral details.     

Sentinel-2 Satellite Image Fusion Methodand Quality Evaluation Analysis

Sentinel-2 satellite sensors acquire three kinds of optical remote sensing images with different spatial resolutions.How to improve the spatial resolution of lower spatial resolution bands by fusion method is one of the problems faced by Sentinel-2 applications.Taking the Sentinel\|2B image as the data source,a high spatial resolution band was generated or selected from the four 10m spatial resolution bands by four methods:the maximum correlation coefficient,the central wavelength nearest neighbor,the pixel maximum and the principal component analysis.We fused the one high spatial resolution band produced and six multispectral bands with 20 m spatial resolution by the five fusion methods of PCA,HPF,WT,GS and Pansharp to produce six multispectral bands with 10 m spatial resolution and the fusion results were evaluated from three aspects:qualitative and quantitative (information entropy,average gradient,spectral correlation coefficient,root mean square error and general image quality index) and classification accuracy of fused images.Results show that the fusion quality of Pansharp with the maximum correlation coefficient is better than other fusion methods,and the classification accuracy is slightly lower than the GS with the pixel maximum of the highest classification accuracy and far higher than the original four multispectral image with 10 m spatial resolution.According to the classification accuracy of experimental data,different fusion methods have different advantages in extraction of different ground objects.In application,appropriate schemes should be selected according to actual research needs.This research can provide reference for Sentinel-2 satellite and similar satellite data processing and application.     

波段自适应细节注入的高分五号与Sentinel-2遥感影像空谱融合

目的 针对当前空谱融合方法应用到高光谱图像融合时,出现的空间细节信息提升明显但光谱失真,或者光谱保真度高但空间细节信息提升不足的问题,本文提出一种波段自适应细节注入的高分五号（GF-5）高光谱图像（30 m）与Sentinel-2多光谱图像（10 m）的遥感影像空谱融合方法。方法 首先,为了解决两个多波段图像不便于直接融合的问题,提出一种波段自适应的融合策略,对多光谱图像波谱范围以外的高光谱图像波段,以相关系数为标准将待融合图像进行分组。其次,针对传统Gram-Schmidt （GS）融合方法用平均权重系数模拟低分辨率图像造成的光谱失真问题,使用最小均方误差估计计算线性拟合系数,再将拟合图像作为第1分量进行GS正变换,提升融合图像的光谱保真度。最后,为了能同时注入更多的空间细节信息,通过非下采样轮廓波变换将拟合图像、空间细节信息图像和多光谱图像的空间、光谱信息融入到重构的高空间分辨率图像中,再将其与其他GS分量一起进行逆变换,最终得到10 m分辨率的GF-5融合图像。结果 通过与当前用于高光谱图像空谱融合的典型方法比较,本文方法对于受时相影响较小的城镇区域,在提升空间分辨率的同时有较好的光谱保真度,且不会出现噪点;对于受时相变化影响大的植被密集区域,本文方法融合图像有较好的清晰度和地物细节信息,且没有噪点出现。本文方法的CC （correlation coefficient）、ERGAS （erreur relative globale adimensionnelle de synthèse）和SAM （spectral angle mapper）相比于传统GS方法分别提升8%、26%和28%,表明本文方法的光谱保真度大大提高。结论 本文方法的结果空间上没有噪点且光谱曲线与原始光谱曲线基本保持一致,是一种兼具高空间分辨率和高光谱保真度的高光谱图像融合方法。     

Nonnegative matrix factorization-based hyperspectral and panchromatic image fusion

The fusion of hyperspectral image and panchromatic image is an effective process to obtain an image with both high spatial and spectral resolutions. However, the spectral property stored in the original hyperspectral image is often distorted when using the class of traditional fusion techniques. Therefore, in this paper, we show how explicitly incorporating the notion of “spectra preservation” to improve the spectral resolution of the fused image. First, a new fusion model, spectral preservation based on nonnegative matrix factorization (SPNMF), is developed. Additionally, a multiplicative algorithm aiming at get the numerical solution of the proposed model is presented. Finally, experiments using synthetic and real data demonstrate the SPNMF is a superior fusion technique for it could improve the spatial resolutions of hyperspectral images with their spectral properties reliably preserved.     

CBERS-02与SPOT5融合数据及其在海岸带土地利用调查中应用能力综合评价

以珠海市横琴岛为试验区,对CBERS-02星多光谱数据和SPOT5全色数据进行融合处理,融合方法包括HIS变换法、Brovey法、主成分融合法、小波融合法、SFIM法、Gram\|Schmidt法、PANSHARP法,并使用定量指标对融合结果进行分析,通过计算各波段均值、信息熵、标准差、相关系数等指标对各融合结果进行分析评价,探讨适合两种数据的融合方法;对CBERS-02和SPOT5数据的PANSHARP法融合数据进行支持向量机分类,总体分类精度为82.55%,Kappa系数为0.7358;试验结果表明,通过融合CBERS\|02多光谱数据与SPOT5全色数据,既可以增强空间纹理信息,又可以弥补SPOT5多光谱数据缺少蓝光波段的缺陷,具有丰富的光谱信息,可以在海岸带土地利用调查广泛应用。     

High-resolution satellite image fusion using regression kriging

Image fusion is an important component of digital image processing and quantitative image analysis. Image fusion is the technique of integrating and merging information from different remote sensors to achieve refined or improved data. A number of fusion algorithms have been developed in the past two decades, and most of these methods are efficient for applications especially for same-sensor and single-date images. However, colour distortion is a common problem for multi-sensor or multi-date image fusion. In this study, a new image fusion method of regression kriging is presented. Regression kriging takes consideration of correlation between response variable (i.e., the image to be fused) and predictor variables (i.e., the image with finer spatial resolutions), spatial autocorrelation among pixels in the predictor images, and the unbiased estimation with minimized variance. Regression kriging is applied to fuse multi-temporal (e.g., Ikonos, QuickBird, and OrbView-3) images. The significant properties of image fusion using regression kriging are spectral preservation and relatively simple procedures. The qualitative assessments indicate that there is no apparent colour distortion in the fused images that coincides with the quantitative checks, which show that the fused images are highly correlated with the initial data and the per-pixel differences are too small to be considered as significant errors. Besides a basic comparison of image fusion between a wavelet based approach and regression kriging, general comparisons with other published fusion algorithms indicate that regression kriging is comparable with other sophisticated techniques for multi-sensor and multi-date image fusion.     

Ehlers pan-sharpening performance enhancement using HCS transform for n-band data sets

The Ehlers fusion method, which combines a standard intensity-hue-saturation (IHS) transform with fast Fourier transform filtering, is a high spectral characteristics preservation algorithm for multitemporal and multisensor data sets. However, for data sets of more than three bands, the fusion process is complicated, because only every three bands are fused repeatedly for multiple times until all bands are fused. The hyper-spherical colour sharpening (HCS) fusion method can fuse a data set with an arbitrary number of bands. The HCS approach uses a transform between an n-dimensional Cartesian space and an n-dimensional hyper-spherical space to get one single intensity component and n ? 1 angles. Moreover, from a structural point of view, the hyper-spherical colour space is very similar to the IHS colour space. Hence, we propose to combine the Ehlers fusion with an HCS transform to fuse n-band data sets with high spectral information preservation, even hyper-spectral images. A WorldView-2 data set including a panchromatic and eight multispectral bands is used for demonstrating the effectiveness and quality of the new Ehlers –HCS fusion. The WorldView-2 image covers different landscapes such as agriculture, forest, water and urban areas. The fused images are visually and quantitatively analysed for spectral preservation and spatial improvement. Pros and cons of the applied fusion methods are related to the analysed different landscapes. Overall, the Ehlers –HCS method shows the efficacy for n-band fusion.     

异源、同源传感器影像融合的比较研究

遥感影像的融合是遥感界的一个研究热点。根据数据源的不同,影像融合可分为异源传感器影像融合和同源传感器影像融合。以TM与SPOT作为异源影像融合的例子,以IKONOS的MS与Pan作为同源影像融合的例子,用5种算法对两种融合类型进行实验与比较。结果表明,同源传感器影像的融合效果好于异源传感器影像的融合效果;不同的融合算法在异源和同源传感器影像融合中的表现不尽相同。SVR变换可同时应用于异源及同源传感器影像的融合,且在提高影像空间分辨率、信息量和清晰度的同时能很好地保持原始多光谱影像的光谱特征。SFIM虽然也可以在两种数据源的融合实验中获得较好的融合效果,但其高频信息融入度最差。MB虽然提高了融合影像的高频信息融入程度,但光谱保真度、信息量和清晰度却不理想。Ehlers适用于异源传感器影像间的融合,而WT则适用于同源传感器影像的融合。     

A novel adaptive fast IHS transform fusion method driven by regional spectral characteristics for Gaofen-2 imagery

ABSTRACT

With the increasing diversity of applications based on the Gaofen-2 satellite imagery, broadly applicable methods to generate high quality fused images is a significant problem to investigate. To obtain an image with high spatial and spectral resolutions from given panchromatic (Pan) images and multispectral (MS) images, most existing fusion algorithms adopt a unified strategy for the whole image. However, regions have distinct characteristics that impact the spatial and spectral resolution processing, on account of their varying regional features. In this article, to satisfy the diverse needs of different regions, a novel fast IHS (Intensity-Hue-Saturation) transform fusion method driven by regional spectral characteristics is proposed to fuse Gaofen-2 imagery. First, by the fast IHS transform framework, the original intensity component is obtained from the upsampled MS imagery. Then, numerous independent regions of upsampled MS imagery are generated by a novel superpixel merging strategy, and the spectral characteristics of these regions are utilized for generating a fusion factor. Next, to acquire a new fused intensity component, the fusion factor is applied to guide the injection of details in the fusion procedure. This fusion factor adapts the method to meet the spatial and spectral resolution needs for each region. Finally, the difference between the new fused intensity component and the original one is regarded as the detail that needs to be injected; these are added equally to the different bands of the upsampled MS imagery to yield the final fused multispectral image. In comparison with other classical algorithms, the visual and statistical analysis reveal that our proposed method can provide better results in improving spatial detail and preserving spectral information.     

Worldview 3全色与短波红外影像两步式融合框架

Worldview 3是目前最先进的高分辨率光学卫星之一。针对Worldview 3遥感卫星数据全色波段和短波红外波段（Short-Wave Infrared,SWIR）空间分辨率差异大、波谱范围不一致所导致的融合结果块状效应以及空间分辨率增强效果有限的问题,提出一种两步式影像融合框架。该框架降低全色波段空间分辨率,实现与SWIR波段的初步融合;将初步融合的结果与原始分辨率全色波段进行第二步融合。通过选择6种典型像素级融合方法两两组合,形成36种融合方案,验证框架的可用性。选取包含植被、建筑、水体等典型地物类型的Worldview 3数据集进行实验,并采用五个定量评价指标进行评价。实验结果表明：使用两步式融合框架进行融合,通过渐进式空间细节注入的方式,避免了直接融合产生的块状效应,实现了短波红外影像的空间分辨率增强;第一步融合采用高通滤波（HPF）融合法,第二步融合采用GS（Gram-Schmidt Transform）变换融合方法,引入的空间信息最多,获得的融合结果质量最好。提出的融合框架既能避免块状效应的产生,又能有效增强SWIR波段的空间分辨率,对于其他卫星的全色与短波红外波段的融合也具有一定借鉴意义。     

基于归一化相关矩的多分辨率遥感图象融合

多传感器数据融合技术已广泛应用于遥感图象处理方面 .针对遥感多光谱图象空间分辨率较低的问题 ,提出了一种基于归一化相关矩的多分辨率图象融合方法 .该方法首先对图象进行二维小波变换 ,然后根据所得到的高频小波系数的一阶、二阶统计特征来定义图象局部灰度相关矩 ,并以此作为图象融合测度来对遥感图象进行多分辨率特征融合 ,从而得到包含更多信息和有效特征的融合图象 .仿真结果表明 ,融合后的图象在保留多光谱信息和提高空间分辨率上均能获得较好的效果 ,因而可以更好地用于目标识别、分类等遥感图象处理方面     

Influence of image fusion approaches on classification accuracy: a case study

While many studies in the field of image fusion of remotely sensed data aim towards deriving new algorithms for visual enhancement, there is little research on the influence of image fusion on other applications. One major application in earth science is land cover mapping. The concept of sensors with multiple spatial resolutions provides a potential for image fusion. It minimises errors of geometric alignment and atmospheric or temporal changes.

This study focuses on the influence of image fusion on spectral classification algorithms and their accuracy. A Landsat 7 ETM+ image was used, where six multispectral bands (30 m) were fused with the corresponding 15 m panchromatic channel. The fusion methods comprise rather common techniques like Brovey, hue‐saturation‐value transform, and principal component analysis, and more complex approaches, including adaptive image fusion, multisensor multiresolution image fusion technique, and wavelet transformation. Image classification was performed with supervised methods, e.g. maximum likelihood classifier, object‐based classification, and support vector machines. The classification was assessed with test samples, a clump analysis, and techniques accounting for classification errors along land cover boundaries. It was found that the adaptive image fusion approach shows best results with low noise content. It resulted in a major improvement when compared with the reference, especially along object edges. Acceptable results were achieved by wavelet, multisensor multiresolution image fusion, and principal component analysis. Brovey and hue‐saturation‐value image fusion performed poorly and cannot be recommended for classification of fused imagery.     

多光谱图象的真实感融合

多光谱信息融合是指将从多光谱探测器获得的同一场景的多光谱信息特征组合到一起，并利用它们在时空上的相关性及信息上的互补性，以利到对景物更全面，清晰的描述，其真实感成像是指融合后的图像能保留原多光谱图像目标在特定条件下的光谱特征，针对用同一种方法来融合不同场景图象存在的问题，通过分析现有融合方法的优缺点和它们适用的场合，提出了一种综合的图象融合方法，该方法对相关性较弱，且互补性明显的图象对，采用一种新的基于人类视觉系统HVS的自适应加权平衡的融合方法来进行融合，这样能突出目标的光谱特性，而对于相关性较强的图象对，则根据场景目标距离的远近，分别采用小波变换和主成份分析来进行融合，实验表明，该方面能适合于不同类型的场景，其得到的融合结果优于用单种方法进行融合的结果，也有利于对目标的探测和识别。     

一种基于IHS和DWT变换的遥感图像融合方法

为了更好地进行不同分辨率图像的融合,提出了一种在Maflab平台上实现基于IHS变换与离散小波变换相结合的多光谱图像与高分辨率图像融合方法.实验结果分析表明,得到的融合图像与原多光谱图像相比,细节信息更为突出,整体信息更为丰富,基本达到了提高融合增强的目的.     

基于加权相关矩的多传感器图像融合方法

针对遥感多光谱图像和全色图像的融合问题,提出了一种基于局部区域特征的多分辨率图像融合方法。该方法首先对图像进行二维小波变换,然后根据高频小波系数的一阶、二阶统计特征来计算加权相关矩,从而得到包含更多信息和有效特征的融合图像。试验结果证明融合图像在保留地物光谱信息和提高空间细节表现能力上都具有很好的效果。     

Covariance intersection based image fusion technique with application to pansharpening in remote sensing

Image fusion of multi-spectral images and panchromatic images has been widely applied to imaging sensors. Multi-spectral images are rich in spectral information whereas panchromatic images have relatively higher spatial resolution. In this paper, we consider the image fusion as an estimation problem, that is to estimate the ideal scene of multi-spectral images at the resolution of panchromatic images. We propose a method of combining the covariance intersection (CI) principle with the expectation maximization (EM) algorithm to develop a novel image fusion approach. In contrast to other fusion methods, the proposed scheme takes cross-correlation among data sources into account, and thus provides consistent and accurate estimates through convex combinations. Since the covariance information is usually unknown in practice, the EM method is employed to provide a maximum likelihood estimate (MLE) of the covariance matrix. Real multi-spectral and panchromatic images are used to evaluate the effectiveness of the proposed EM-CI method. The proposed algorithm is found to preserve both the spectral information of the multi-spectral image and the high spatial resolution information of the panchromatic image more effectively than the conventional image fusion techniques.     

运动目标检测的红外与可见光图像融合方法

在弱可见光条件下,对同一场景监控的红外与可见光图像进行融合,使融合图像即显示红外目标,又能保留可见光图像的细节结构信息,方便观察者对场景的观察与监控。充分利用红外成像的特点,热目标与背景的温度差会使目标在红外图像中的灰度值更大。使用红外序列建立稳定的背景模型,当前帧与背景的差得到运动目标区域,然后,将目标区域内的红外目标融合到可见光图像中,达到对红外运动目标检测的目的。     

Image Fusion Based on NSCT and Sparse Representation for Remote Sensing Data

The practice of integrating images from two or more sensors collected from the same area or object is known as image fusion. The goal is to extract more spatial and spectral information from the resulting fused image than from the component images. The images must be fused to improve the spatial and spectral quality of both panchromatic and multispectral images. This study provides a novel picture fusion technique that employs L0 smoothening Filter, Non-subsampled Contour let Transform (NSCT) and Sparse Representation (SR) followed by the Max absolute rule (MAR). The fusion approach is as follows: first, the multispectral and panchromatic images are divided into lower and higher frequency components using the L0 smoothing filter. Then comes the fusion process, which uses an approach that combines NSCT and SR to fuse low frequency components. Similarly, the Max-absolute fusion rule is used to merge high frequency components. Finally, the final image is obtained through the disintegration of fused low and high frequency data. In terms of correlation coefficient, Entropy, spatial frequency, and fusion mutual information, our method outperforms other methods in terms of image quality enhancement and visual evaluation.     

Block-regression based fusion of optical and SAR imagery for feature enhancement

This paper focuses on the establishment of a pixel-level fusion framework for optical and synthetic aperture radar (SAR) images to combine these two types of remotely sensed imagery for feature enhancement. We have proposed a new fusion technique, namely block-based synthetic variable ratio (Block-SVR), which is a technique based on multiple linear regression of block regions to fuse optical and SAR imagery. In order to investigate the effectiveness of the method, the fusion results of a higher resolution airborne SAR image and a lower resolution multispectral image are presented. According to the fusion results, the fused images have enhanced certain features, namely the spatial and textural content and features that are invisible in multispectral images, while preserving colour characteristics. The spectral, spatial and textural effects of the presented algorithm were also evaluated mainly by visual and quantitative methods, and compared to those of intensity-hue-saturation (IHS), principal component analysis (PCA) and wavelet-based methods. During the implementation of the block-regression based technique there are at least two advantages. One is that the block-regression based technique drastically decreases the amount of computation, whereas regression of the whole scene image is almost impossible. The other, most important, advantage is that adjustment of regressed block size can result in different emphasis between preservation of spectral characteristics and enhancement of spatial and textural content. The larger the regression block, the more the spatial and textural details are enhanced. In contrast, the smaller the regression block, the more the spectral features are preserved. The assessments indicate that the block-regression based method is more flexible than others, because it can achieve a satisfactory trade-off between preservation of spectral characteristics and enhancement of spatial and textural content by selection of optimal block size with respect to visual interpretation and mapping. This paper also proposes a scheme for the fusion of SPOT5 panchromatic, XS images with airborne SAR images using the block-regression based technique.