Multi-resolution analysis techniques and nonlinear PCA for hybrid pansharpening applications

Hyperspectral images have a higher spectral resolution (i.e., a larger number of bands covering the electromagnetic spectrum), but a lower spatial resolution with respect to multispectral or panchromatic acquisitions. For increasing the capabilities of the data in terms of utilization and interpretation, hyperspectral images having both high spectral and spatial resolution are desired. This can be achieved by combining the hyperspectral image with a high spatial resolution panchromatic image. These techniques are generally known as pansharpening and can be divided into component substitution (CS) and multi-resolution analysis (MRA) based methods. In general, the CS methods result in fused images having high spatial quality but the fused images suffer from spectral distortions. On the other hand, images obtained using MRA techniques are not as sharp as CS methods but they are spectrally consistent. Both substitution and filtering approaches are considered adequate when applied to multispectral and PAN images, but have many drawbacks when the low-resolution image is a hyperspectral image. Thus, one of the main challenges in hyperspectral pansharpening is to improve the spatial resolution while preserving as much as possible of the original spectral information. An effective solution to these problems has been found in the use of hybrid approaches, combining the better spatial information of CS and the more accurate spectral information of MRA techniques. In general, in a hybrid approach a CS technique is used to project the original data into a low dimensionality space. Thus, the PAN image is fused with one or more features by means of MRA approach. Finally the inverse projection is used to obtain the enhanced image in the original data space. These methods, permit to effectively enhance the spatial resolution of the hyperspectral image without relevant spectral distortions and on the same time to reduce the computational load of the entire process. In particular, in this paper we focus our attention on the use of Nonlinear Principal Component Analysis (NLPCA) for the projection of the image into a low dimensionality feature space. However, if on one hand the NLPCA has been proved to better represent the intrinsic information of hyperspectral images in the feature space, on the other hand an analysis of the impact of different fusion techniques applied to the nonlinear principal components in order to define the optimal framework for the hybrid pansharpening has not been carried out yet. More in particular, in this paper we analyze the overall impact of several widely used MRA pansharpening algorithms applied in the nonlinear feature space. The results obtained on both synthetic and real data demonstrate that an accurate selection of the pansharpening method can lead to an effective improvement of the enhanced hyperspectral image in terms of spectral quality and spatial consistency, as well as a strong reduction in the computational time.     

Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images

In this paper, we propose an optimum algorithm, in the minimum mean-square-error (mmse) sense, for panchromatic (Pan) sharpening of very high resolution multispectral (MS) images. The solution minimizes the squared error between the original MS image and the fusion result obtained by spatially enhancing a degraded version of the MS image through a degraded version, by the same scale factor, of the Pan image. The fusion result is also optimal at full scale under the assumption of invariance of the fusion parameters across spatial scales. The following two versions of the algorithm are presented: a local mmse (lmmse) solution and a fast implementation which globally optimizes the fusion parameters with a moderate performance loss with respect to the lmmse version. We show that the proposed method is computationally practical, even in the case of local optimization, and it outperforms the best state-of-the-art Pan-sharpening algorithms, as resulted from the IEEE Data Fusion Contest 2006, on true Ikonos and QuickBird data and on simulated Pleiades data.     

Comprehensive review on fusion techniques for spatial information enhancement in hyperspectral imagery

The volume of data grows with the advent of multiple types of remote sensing sensors and in order to extract the most useful information there is a need to combine the data gathered from the different sources. The widely used panchromatic and multispectral imageries in many applications offer decimetric and metric spatial resolution. However, the spectral resolution of these images is poor. Hyperspectral imaging has unique characteristics of providing very fine spectral resolution in a large number of bands with decametric spatial resolution and found to be highly useful for a wide span of application areas that requires high spectral resolution. The fusion of spectral and spatial information provides an effective way of enhancing the spatial quality of hyperspectral imagery as well as a method for preserving spectral quality. This fusion process is not a trivial task as always there has been a tradeoff between the preservation of spatial and spectral quality information as in the original sources of fusion. In this paper, a review on hyperspectral pansharpening and hyperspectral multispectral fusion based approaches has been reported. The widely adopted quantitative and qualitative performance measures to verify the fusion results are highlighted. In addition, the challenges in existing fusion techniques have also been discussed.     

High spectral quality pansharpening approach based on MTF-matched filter banks

Pansharpening consists in merging a low-resolution multispectral image (MS) with a high spatial resolution panchromatic image (PAN) to produce a high resolution pansharpened MS image. It consists in enhancing spatially the low-resolution MS image by injecting the missing details provided by the high-resolution PAN image. In this paper, we propose a novel pansharpening approach based on decomposition/reconstruction processing using low-pass and high-pass filter banks. On the one hand, the low-pass approximation (taking into account the imaging system modulation transfer function MTF) of the pansharpened MS image is assumed to be equal to the original MS image in order to preserve the spectral quality. On the other hand, the high-pass filter allowing us to extract the high-frequency PAN details is designed as complementary filter to the low-pass one in order to provide perfect reconstruction in the ideal case. Quantitative assessment performed on reduced and full-resolution images are used to validate the proposed technique and compare it to state-of-art. Experimental results using Pléaides and GeoEye-1 data show that our proposed fusion schema outperforms the pre-existing methods visually as well as quantitatively.     

基于数据融合估计理论的Kalman滤波

本文在经典Kalman滤波算法的基础上针对多传感器数据处理的特点,根据不同的数据融合结构方案给出了多传感器数据融合系统的Kalman滤波算法。根据多传感器数据融合估计知识,建立了基于Kalman滤波的数据融合模型;给出了融合结构的数据融合滤波算法：集中式Kalman滤波算法;进行了算法仿真演示实验。实验结果表明,通过多传感器融合能够提高系统的估计精度。     

基于多级特征补偿的遥感图像时空融合方法

在许多地球科学应用中要用到大量的高时空分辨力的地球观测数据。时空图像融合方法为产生高时空分辨力的数据提供了一种可行且经济的解决方案。然而,现有的一些基于学习的方法对于图像深层特征提取能力较弱,对于高分辨力图像细节特征利用度不够。针对这些问题,提出一种基于多级特征补偿的遥感图像时空融合方法。该方法使用2个分支进行多层级的特征补偿,并提出了融合通道注意力机制的残差模块作为网络的基本组成单元,可以将高分辨力输入图像的深层特征更为详尽地提取利用。提出一种基于拉普拉斯算子的边缘损失,在节省预训练计算开销的同时取得了很好的融合效果。使用从山东和广东2个地区采集的Landsat和中分辨力成像光谱仪(MODIS)卫星图像对所提出的方法进行实验评估。实验结果表明,提出的方法在视觉外观和客观指标方面都具有更高质量。     

红外与可见光图像融合质量评价方法综述

图像融合质量评价标准是目前图像融合中急需解决的问题之一。在不同的应用场景中,选择适当的融合算法对现有融合算法进行改进以及研究新的融合算法都是至关重要的。综述了无参考图像的客观质量评价方法,重点介绍了基于结构信息的图像融合质量评价方法,最后探讨了图像融合质量评价方法的发展趋势。     

自适应字典学习的卷积稀疏表示遥感图像融合

基于细节注入方案的遥感图像融合主要包括两个步骤:空间细节提取与注入。为保证被提取细节的质量与确定合适的调制系数,本文提出一种基于自适应字典学习的卷积稀疏表示遥感图像融合方法。该方法先利用引导滤波和非抽取小波变换来分别获取全色图像和多光谱图像的空间细节;然后自适应地学习提取空间细节的字典,并将其引入卷积稀疏表示模型来重构联合细节图像;最后,将联合细节通过联合判别调制系数注入到上采样的多光谱图像中得到最终融合结果。实验结果表明,本文方法的融合结果无论从主观效果还是客观定量评价,都优于一些主流的遥感图像融合方法。      

小波域结构相似度融合图像质量评价方法

针对融合图像质量评价问题,分析了图像质量评价与融合图像质量评价的关系,给出了融合图像质量评价方法的一般表达公式,指出构造实际并不存在的参考图像是解决融合图像质量评价问题的关键。在此基础上,基于空域结构相似度评价方法,对输入源图像和融合图像分别进行小波分解,利用输入源图像小波分解系数构造参考图像小波系数,然后根据人眼视觉敏感度带通特性对参考图像和融合图像的各小波频带进行加权,从而得到整幅图像的小波域结构相似度评价指标,利用目标可探测性、细节可分辨能力和图像整体舒适性构成主观评价指标分别和交互信息量、基于空域的结构相似度比较。实验结果表明,相比于传统的客观评价方法,提出的方法所得结果与主观评价结果的一致性更好。     

基于变分的多尺度遥感图像融合算法

全色图像与多光谱图像融合时,忽略了上采样的重要性和通道间细节的差异性.针对前者,利用不同尺度下自相似块,估计出低分辨率图像丢失信息,从而修改了图像上采样的策略,并以此构造目标函数的保真项;针对后者,利用全色图像和光谱图像梯度域结构相似性,提出局部加权动态稀疏约束,构造目标函数的正则项.本文基于变分法理论,构造了新的目标函数,并提出了多尺度迭代融合框架,通过多次迭代逐步提高融合图像的分辨率,每一层的融合结果更加准确,从而提高最终的融合精度.本文算法与Brovey等成分替代算法、P+XS等变分算法、MTF_GLP等多分辨分析算法进行比较.实验结果表明,本算法的融合结果具有良好的视觉效果,且在客观评价指标上比所有对比算法的最优值平均值均有提高.     

Underwater imaging with a moving acoustic lens

The acoustic lens is a high-resolution, forward-looking sonar for three dimensional (3-D) underwater imaging. We discuss processing the lens data for recreating and visualizing the scene. Acoustical imaging, compared to optical imaging, is sparse and low resolution. To achieve higher resolution, we obtain a denser sample by mounting the lens on a moving platform and passing over the scene. This introduces the problem of data fusion from multiple overlapping views for scene formation, which we discuss. We also discuss the improvements in object reconstruction by combining data from several passes over an object. We present algorithms for pass registration and show that this process can be done with enough accuracy to improve the image and provide greater detail about the object. The results of in-water experiments show the degree to which size and shape can be obtained under (nearly) ideal conditions.     

shearlet变换和区域特性相结合的图像融合

为了提高多模医学图像或多聚焦图像的融合性能,结合shearlet变换能够捕捉图像细节信息的性质,提出了一种基于shearlet变换的图像融合算法。首先,用shearlet变换将已精确配准的两幅原始图像分解,得到低频子带系数和不同尺度不同方向的高频子带系数。低频子带系数使用改进的加权融合算法,用平均梯度来计算加权参量,以此来改善融合图像轮廓模糊度高的问题,高频子带系数采用区域方差和区域能量相结合的融合规则,以得到丰富的细节信息。最后,进行shearlet逆变换得到融合图像。结果表明,此算法在主观视觉效果和客观评价指标上优于其它融合算法。     

可见光与红外融合的汽车抗晕光图像评价方法

为了解决汽车抗晕光场景中,融合图像质量的客观评价与人眼视觉效果不一致的问题,提出了一种新的可见光与红外融合图像质量评价方法,该方法通过设计自适应迭代阈值法自动确定融合图像的晕光临界灰度值,并将融合图像自动分为晕光区和非晕光区。针对晕光区,设计晕光消除度指标评价晕光消除的效果;针对非晕光区,从多方面评价色彩、细节信息的增强效果,并甄选出合适的指标构成完整的图像质量评价体系。为验证该方法的合理性,对采用4种不同算法的融合图像进行评价,实验分析表明,该方法的主客观评价结果一致,适于评判不同可见光与红外融合的抗晕光图像质量及算法的优劣。     

基于ResNet的红外与低照度可见光图像融合方法

针对传统红外与低照度可见光图像融合后,容易造成目标模糊不清、细节信息缺失等问题,本文提出一种低照度可见光图像预增强与残差网络(Residual Network,ResNet)相结合的图像融合方法。该方法首先利用单尺度Retinex(Single Scale Retinex,SSR)算法对低照度可见光图像进行增强预处理,得到增强的可见光图像。其次,利用ResNet-50分别从增强后的可见光图像和红外图像中提取深度特征。然后,采用L1范数对生成的深度特征进行正则化处理,并通过上采样操作将其分辨率恢复至输入图像大小,得到权重图。最后,使用加权平均策略获取融合图像。实验结果表明,本文算法能更好地保留输入图像的纹理细节和结构信息;使用TNO数据集与现有的三种典型算法对比,该算法融合结果的离散余弦特征互信息(FMIdct)、小波特征互信息(FMIw)、基于噪声评估的融合性能(Nabf)、结构相似度测量(SSIM)四种客观指标总体优于对比算法。     

基于热辐射信息保留的图像融合算法

针对现有的红外与可见光图像融合算法无法很好地保留红外图像热辐射信息这一问题,提出了一种基于热辐射信息保留的图像融合算法。通过NSCT（non-subsampled contourlet transform）变换对红外与可见光图像进行多尺度分解,得到各自的高频子带和低频子带,可见光低频子带部分经拉普拉斯算子提取特征后与红外低频子带部分叠加得到融合图像的低频系数,高频部分使用基于点锐度和细节增强的融合规则进行融合以得到高频系数,最后通过逆NSCT变换重构得到融合图像。实验表明,相较于其它图像融合算法,所提算法能在保留红外图像热辐射信息的同时,保有较好的清晰细节表现能力,并在多项客观评价指标上优于其它算法,具有更好的视觉效果,且在伪彩色变换后有良好的视觉体验,验证了所提算法的有效性和可行性。     

基于多小波分析的图像融合算法

从电磁波频谱带宽的角度,提出了图像融合的一种物理解释。介绍了多小波变换的一般理论,描述了多小波图像融合的方法。用GHM多小波、CL多小波及Db4单小波对可见光与红外图像进行了融合。利用平均熵作为判定准则,比较了GHM多小波、CL多小波及Db4单小波的融合效果,实验结果表明CL多小波算法效果最好。     

基于FCM与引导滤波的红外与可见光图像融合

针对传统红外与可见光图像融合算法中存在的目标模糊、细节丢失、算法不稳定等问题,提出了一种基于模糊C均值聚类（Fuzzy C-means, FCM）与引导滤波的红外与可见光图像融合方法。原图像经过非下采样剪切波变换（Nonsubsampled Shearlet Transform, NSST）后对低频子带进行引导滤波增强,再利用FCM与双通道脉冲发放皮层模型（Dual Channel Spiking Cortical Model, DCSCM）结合对高低频子带进行融合,最后经NSST逆变换得到融合图像。实验结果表明,本文算法稳定,主观评价上所得融合图像目标明确,细节保留较为完整,客观评价上在标准差、互信息、平均梯度、信息熵和边缘保留因子等评价标准中表现优良。     

基于MSPCNN与FCM的红外与可见光图像融合

针对红外和可见光图像融合存在的轮廓信息不全、边缘及纹理细节信息缺失等问题,提出一种改进简化脉冲耦合神经网络（Improved Simplified Pulse Coupled Neural Network, MSPCNN）和模糊C-均值（Fuzzy C-mean, FCM）图像融合算法。首先,将红外和可见光图像用非下采样剪切波算法（NonSubsampled Shearlet Transform,NSST）分解为高低频子带;然后对分解后的高频子带采用MSPCNN融合,用一种高斯分布权重矩阵进行处理,增强细节信息和对比度;接着,将得到的低频子带图像使用FCM聚类算法进行聚类中心提取,设置聚类中心近似阈值简化过程,实现背景分类提取;最后利NSST进行逆变换,从而完成红外和可见光的图像融合过程。通过客观评价指标计算,本文所提方法在平均梯度、标准差、平均相似度等参考指标上相对于其他同类型算法均有改善提高,由于模型参数的简化,算法运行速度相对于其他算法得到提升,算法更适用于复杂场景。     

基于联合卷积分析与合成稀疏表示的遥感图像分量替换融合方法

针对遥感图像融合中传统分量替换方法光谱失真严重问题,提出了一种基于联合卷积分析与合成稀疏表示的改进分量替换融合方法.与传统分量替换方法不同,该方法旨在改进融合过程中空间细节信息提取和注入策略,以生成具有更高光谱与空间质量的遥感图像.首先利用联合卷积分析与合成稀疏表示算法分别对强度分量和直方图匹配后的全色图像进行分解,获...     

基于非下采样Contourlet变换的多传感器图像融合

根据非下采样Contourlet变换同时具有多尺度多分辨分析和平移不变性质的特点,提出一种基于非下采样Contourlet变换的多传感器图像融合方法,将其应用于多传感器图像融合的两个重要领域——多聚焦图像融合和高分辨、多光谱图像融合,从视觉效果和信息量指标方面对融合图像进行主观评判和数值评价.实验中将本文方法与Contourlet变换、小波变换、主成分分析等方法进行了比较,结果表明本文方法得到的融合结果具有更优的视觉质量和量化指标,能很好地将源图像的细节信息融合在一起,拓广了NSCT的应用范围.