Pan-sharpening via multi-scale and multiple deep neural networks

Interpreting remote sensing images by combining manual visual interpretation and computer automatic classification and recognition is an important application of human–computer interaction (HCI) in the field of remote sensing. Remote sensing images with high spatial resolution and high spectral resolution is an important basis for automatic classification and recognition. However, such images are often difficult to obtain directly. In order to solve the problem, a novel pan-sharpening method via multi-scale and multiple deep neural networks is presented. First, the non-subsampled contourlet transform (NSCT) is employed to decompose the high resolution (HR)/low resolution (LR) panchromatic (PAN) images into the high frequency (HF)/low frequency (LF) images, respectively. For pan-sharpening, the training sets are only sampled from the HF images. Then, the DNN is utilized to learn the feature of the HF images in different directions of HR/LR PAN images, which is trained by the image patch pair sampled from HF images of HR/LR PAN images. Moreover, in the fusion stage, NSCT is also employed to decompose the principal component of initially amplified LR multispectral (MS) image obtained by the transformation of adaptive PCA (A-PCA). The HF image patches of LR MS, as the input data of the trained DNN, go through forward propagation to obtain the output HR MS image. Finally, the output HF sub-band images and the original LF sub-band images of LR MS image fuse into a new sub-band set. The inverse transformations of NSCT and A-PCA , residual compensation are conducted to obtain the pan-sharpened HR MS. The experimental results show that our method is better than other well-known pan-sharpening methods.     

Estimation of the Number of Decomposition Levels for a Wavelet-Based Multiresolution Multisensor Image Fusion

The wavelet-based scheme for the fusion of multispectral (MS) and panchromatic (PAN) imagery has become quite popular due to its ability to preserve the spectral fidelity of the MS imagery while improving its spatial quality. This is important if the resultant imagery is used for automatic classification. Wavelet-based fusion results depend on the number of decomposition levels applied in the wavelet transform. Too few decomposition levels result in poor spatial quality fused images. On the other hand, too many levels reduce the spectral similarity between the original MS and the pan-sharpened images. If the shift-invariant wavelet transform is applied, each excessive decomposition level results in a large computational penalty. Thus, the choice of the number of decomposition levels is significant. In this paper, PAN and MS image pairs with different resolution ratios were fused using the shift-invariant wavelet transform, and the optimal decomposition levels were determined for each resolution ratio. In general, it can be said that the fusion of images with larger resolution ratios requires a higher number of decomposition levels. This paper provides the practitioner an understanding of the tradeoffs associated with the computational demand and the spatial and spectral quality of the wavelet-based fusion algorithm as a function of the number of decomposition levels     

High spectral quality pansharpening approach based on MTF-matched filter banks

Pansharpening consists in merging a low-resolution multispectral image (MS) with a high spatial resolution panchromatic image (PAN) to produce a high resolution pansharpened MS image. It consists in enhancing spatially the low-resolution MS image by injecting the missing details provided by the high-resolution PAN image. In this paper, we propose a novel pansharpening approach based on decomposition/reconstruction processing using low-pass and high-pass filter banks. On the one hand, the low-pass approximation (taking into account the imaging system modulation transfer function MTF) of the pansharpened MS image is assumed to be equal to the original MS image in order to preserve the spectral quality. On the other hand, the high-pass filter allowing us to extract the high-frequency PAN details is designed as complementary filter to the low-pass one in order to provide perfect reconstruction in the ideal case. Quantitative assessment performed on reduced and full-resolution images are used to validate the proposed technique and compare it to state-of-art. Experimental results using Pléaides and GeoEye-1 data show that our proposed fusion schema outperforms the pre-existing methods visually as well as quantitatively.     

Practical remote sensing image fusion method based on guided filter and improved SML in the NSST domain

Due to the different characteristics of image modality, the panchromatic (PAN) and multispectral (MS) images include complementary and redundancy information in the spatial and spectral resolutions. Image fusion is an effective way to integrate the source PAN and MS images to obtain high-resolution MS image. In this paper, a novel remote sensing image fusion scheme in non-subsample Shearlet transform (NSST) domain is presented. An enhancement strategy is designed to solve the insufficiency of spatial detail in multiresolution analysis (MRA)-based methods after the intensity–hue–saturation (IHS) color space transform. Then, in the NSST fusion process, a guided filter-based low-frequency coefficient fusion rule and an improved sum-modified-Laplacian (SML)-based high-frequency coefficient fusion rule are proposed. The final fused image can be obtained through the inverse NSST transform and inverse IHS transform. Two different groups of satellite dataset are utilized to evaluate the fusion performance. The experiment results demonstrate that the proposed approach can achieve more spatial details and less spectral distortion compared with the existing methods regarding both the visual quality and the objective measurements.     

两帧图像的"亚象元"技术提高探测图像的空间分辨率

给出两帧图像“亚象元”成像系统的调制传递函数 ,由此分析得出 ,这种技术将采样频率提高 1.4倍 ,适用于探测器占孔比接近 1,按照“亚象元”数字图像的特征 ,进行三种不同插值方法的研究。模拟“亚象元”成像证明 ,两帧图像“亚象元”技术能使图像的空间分辨率提高 1.3 3倍 ,再经过插值处理后 ,图像的空间分辨率进一步提高 ,接近于 2倍的理论值 ,图像质量略逊于四帧图像的亚象元级像。以一块分辨率板作为目标物的实验研究表明 ,两帧图像的“亚象元”成像 ,配合去卷积、图像增强以及插值等数字图像处理 ,空间分辨率得到提高     

基于Contourlet系数局部特征的选择性遥感图像融合算法

为了使融合后的多光谱图像在显著提高空间分辨率的同时,尽可能多地保持原始多光谱特性,提出了一种基于Contourlet变换系数局部特征的选择性遥感图像融合方法。根据多光谱和全色图像融合过程中Contourlet变换后的低频和高频部分融合目的的不同,对得到的近似和各层各方向的细节分量分别运用窗口邻域移动模板逐一计算相应区域Contourlet系数阵的不同局部特征量,然后选择适当的准则,对图像的近似和细节分量分别应用不同的策略在Contourlet系数域内进行选择性融合,通过Contourlet和亮度-色调-饱和度(IHS)逆变换得到融合的高分辨率多光谱图像。采用Landsat TM多光谱和SPOT全色图像进行的融合实验结果表明:提出的算法在显著提高空间分辨率的同时,又能很好地保持原始图像的光谱特征,并优于传统的融合方法。     

基于艇载遥感成像数据的高空间分辨率全色遥感图像仿真

高空间分辨率全色遥感图像在军事侦察、地面监视等领域具有较高的应用价值.为模拟星载全色遥感图像, 提出了一种由艇载遥感成像系统获取的低空遥感图像为数据源的高空间分辨率全色遥感图像仿真方法.首先将低空宽视场图像按典型地物类型进行监督分类, 其次将低空宽视场图像与多光谱图像按不同地物类型分类拟合, 并将多光谱拟合结果合成高空间分辨率全色仿真图像, 最后对高空间分辨率全色仿真图像进行仿真精度评价.相比星载全色遥感图像, 仿真图像同样具备高空间分辨率、全色波段、宽视场等特点.仿真方法可为星载全色遥感图像仿真提供较准确的数据支撑.     

On the spatial resolution of the IR remote sensing systems

According to the close relation between spatial resolution and radiant resolution and utilizing the experimental equation of MRTD, it is proved that there is a selectable optimum value of IFOV for particular spatial frequency or spatial frequency region which is interesting. The convolution process of line scanner is also analyzed, it is concluded that the spatial resolution can greatly exceed the limit of IFOV as long as the contrast is large enough, especially for linear or spot targets. Thus, the relative phenomena of the satellite image can be explained effectively. It is indicated that the spatial resolution can further be improved, if the processing of solving convolution operation is carried out in the digitized image processing.     

Multi-resolution analysis techniques and nonlinear PCA for hybrid pansharpening applications

Hyperspectral images have a higher spectral resolution (i.e., a larger number of bands covering the electromagnetic spectrum), but a lower spatial resolution with respect to multispectral or panchromatic acquisitions. For increasing the capabilities of the data in terms of utilization and interpretation, hyperspectral images having both high spectral and spatial resolution are desired. This can be achieved by combining the hyperspectral image with a high spatial resolution panchromatic image. These techniques are generally known as pansharpening and can be divided into component substitution (CS) and multi-resolution analysis (MRA) based methods. In general, the CS methods result in fused images having high spatial quality but the fused images suffer from spectral distortions. On the other hand, images obtained using MRA techniques are not as sharp as CS methods but they are spectrally consistent. Both substitution and filtering approaches are considered adequate when applied to multispectral and PAN images, but have many drawbacks when the low-resolution image is a hyperspectral image. Thus, one of the main challenges in hyperspectral pansharpening is to improve the spatial resolution while preserving as much as possible of the original spectral information. An effective solution to these problems has been found in the use of hybrid approaches, combining the better spatial information of CS and the more accurate spectral information of MRA techniques. In general, in a hybrid approach a CS technique is used to project the original data into a low dimensionality space. Thus, the PAN image is fused with one or more features by means of MRA approach. Finally the inverse projection is used to obtain the enhanced image in the original data space. These methods, permit to effectively enhance the spatial resolution of the hyperspectral image without relevant spectral distortions and on the same time to reduce the computational load of the entire process. In particular, in this paper we focus our attention on the use of Nonlinear Principal Component Analysis (NLPCA) for the projection of the image into a low dimensionality feature space. However, if on one hand the NLPCA has been proved to better represent the intrinsic information of hyperspectral images in the feature space, on the other hand an analysis of the impact of different fusion techniques applied to the nonlinear principal components in order to define the optimal framework for the hybrid pansharpening has not been carried out yet. More in particular, in this paper we analyze the overall impact of several widely used MRA pansharpening algorithms applied in the nonlinear feature space. The results obtained on both synthetic and real data demonstrate that an accurate selection of the pansharpening method can lead to an effective improvement of the enhanced hyperspectral image in terms of spectral quality and spatial consistency, as well as a strong reduction in the computational time.     

NSCT域内基于改进PCNN和区域能量的多光谱和全色图像融合方法

针对多光谱和全色图像的融合,提出了一种NSCT域内基于改进脉冲耦合神经网络（PCNN）和区域能量的融合方法。首先,利用NSCT将图像分解为一个低频子带和多个不同方向的带通子带。然后,对分解后的低频子带采用基于区域能量的自适应加权算法进行融合;在带通方向子带,结合改进的脉冲耦合神经网络,使用带通方向子带系数作为PCNN的外部输入激励,经过PCNN点火获得待融合图像的点火映射图,根据点火时间计算点火映射图的区域能量,通过判决算子选择待融合图像的带通方向子带系数作为融合系数。最后,对融合处理后的NSCT变换系数进行重构生成融合图像。实验结果显示:在迭代次数为100次时,与改进小波算法相比,标准差提高了9.48%,熵提高了0.95%,相关系数提高了21.56%,偏差指数降低了29.66%;与Contourlet算法相比,标准差提高了9.73%,熵提高了0.94%,相关系数提高了11.27%,偏差指数降低了9.45%;与NSCT算法相比,标准差提高了3.84%,熵提高了3.34%,相关系数提高了7.89%,偏差指数降低了7.42%。     

Generalized IHS‐Based Satellite Imagery Fusion Using Spectral Response Functions

Image fusion is a technical method to integrate the spatial details of the high‐resolution panchromatic (HRP) image and the spectral information of low‐resolution multispectral (LRM) images to produce high‐resolution multispectral images. The most important point in image fusion is enhancing the spatial details of the HRP image and simultaneously maintaining the spectral information of the LRM images. This implies that the physical characteristics of a satellite sensor should be considered in the fusion process. Also, to fuse massive satellite images, the fusion method should have low computation costs. In this paper, we propose a fast and efficient satellite image fusion method. The proposed method uses the spectral response functions of a satellite sensor; thus, it rationally reflects the physical characteristics of the satellite sensor to the fused image. As a result, the proposed method provides high‐quality fused images in terms of spectral and spatial evaluations. The experimental results of IKONOS images indicate that the proposed method outperforms the intensity‐hue‐saturation and wavelet‐based methods.     

一种基于非采样Contourlet变换的遥感图像融合算法

为使融合后的多光谱图像尽可能保持原多光谱图像光谱特性的同时提高空间质量,提出了一种基于非采样Contourlet变换（NSCT）和多尺度边缘检测的融合算法。介绍了非采样Contourlet变换和多尺度边缘检测;设计了基于多尺度边缘检测、直接替代的高频、低频子带融合规则;用QuickBird卫星高分辨率遥感图像进行仿真实验。实验结果表明该算法能够在保持光谱信息的同时注入更丰富的空间细节信息,优于传统的Wavelet变换法和Contourlet变换法。     

基于KL与小波联合变换的多光谱图像压缩

鉴于卫星拍摄的遥感图像的空间分辨率和光谱分辨率越来越高,在一些应用中,常会对多光谱图像进行压缩。为了提高多光谱图像的压缩质量,提出了联合相位相关和仿射变换的图像配准方法,有效提高了图像谱段之间的相关性。针对多光谱图像压缩,提出了结合Karhunen-Love,KL变换去除谱间相关和嵌入式二维小波编码方法。相比JPEG2000谱段图像独立压缩方法,提出方法解压图像的Peak Signal to Noise Ratio,PSNR值平均提高2.1 dB。实验结果表明:所提出的方法能在相同的压缩率下获得比JPEG2000谱段图像独立压缩方法更好的图像质量。     

基于视觉注意力机制的多源遥感图像语义分割

近年来,随着空间感知技术的不断发展,对多源遥感图像的融合处理需求也逐渐增多,如何有效地提取多源图像中的互补信息以完成特定任务成为当前的研究热点。针对多源遥感图像融合语义分割任务中,多源图像的信息冗余和全局特征提取难题,本文提出一种将多光谱图像（Multispectral image, MS）、全色图像（Panchromatic image, PAN）和合成孔径雷达 （Synthetic Aperture Radar, SAR）图像融合的基于Transformer的多源遥感图像语义分割模型Transformer U-Net （TU-Net）。该模型使用通道交换网络（Channel-Exchanging-Network, CEN）对融合支路中的多源遥感特征图进行通道交换,以获得更好的信息互补性,减少数据冗余。同时在特征图拼接后通过带注意力机制的Transformer模块对融合特征图进行全局上下文建模,提取多源遥感图像的全局特征,并以端到端的方式分割多源图像。在MSAW数据集上的训练和验证结果表明,相比目前的多源融合语义分割算法,在F₁值和Dice系数上分别提高了3.31%~11.47%和4.87%~8.55%,对建筑物的分割效果提升明显。      

Mining discriminative spatial cues for aerial image quality assessment towards big data

Evaluating massive-scale aerial/satellite images quality is useful in computer vision and intelligent applications. Traditional local features-based algorithms have achieved impressive performance. However, spatial cues, i.e., geometric property and topological structure, have not been exploited effectively and explicitly. Thus, in this paper, we propose a novel method for image quality assessment towards aerial/satellite images, where discriminative spatial cues are well encoded. More specifically, in order to mine inherent spatial structure of aerial images, each image is segmented into several basic components such as buildings, airport and playground. Afterwards, a weighted region adjacency graph (RAG) is built based on the basic components to represent the spatial feature of each aerial image. We integrate the spatial feature with other transform domain features, and train a support vector regression model to achieve image quality assessment. Experiments demonstrate that our method shows competitive or even better performance compared with several state-of-the-art algorithms.     

Fusion of hyperspectral and panchromatic images with guided filter

Guided filter has been widely used in image fusion. However, most of the guided filter-based fusion methods generate the spatial detail image by making a compromise between the spatial detail of the panchromatic (PAN) and that of the hyperspectral (HS) intensity component. The intensity component cannot well present the edge and texture features of the HS image. The spectral distortion usually occurs due to the injected redundant spatial detail. To overcome this problem, this study presents a novel HS image fusion method by taking the advantage of the guided filter. The characteristics of the PAN and HS images are simultaneously considered. The guided filter is employed to generate the spatial detail image of each HS image band successively. The generated spatial detail image is further optimized by minimizing the difference between each band of the spatial detail image and its corresponding band of the HS image, with the help of a novel injection gains matrix. Experiments performed on various satellite datasets demonstrate that the superiority of the proposed method in spectral maintenance and spatial quality aspects.     

气象卫星红外云图与水汽图的融合技术研究

将多分辨率分析融合方法和多尺度几何分析融合方法应用于气象卫星水汽图和红外云图的融合中,并用主观视觉、平均互信息和Xydeas-Petrovic指标对各种融合算法的性能分别进行了定性和定量评价。结果表明,与源图像相比,融合图像取得了更好的视觉效果,图像中包含了更多的信息量,云体清晰度和云的层次感得到了提高,纹理变得细致了。从平均互信息和Xydeas-Petrovic指标看,多尺度几何分析融合方法的效果较多分辨率分析融合方法更好。     

一种Enlighten-GAN网络的指纹超分辨率重建方法北大核心CSCD

将低分辨率(LR)图像重建为高分辨率(HR)图像的主流模型是生成对抗网络(GAN)。然而,由于基于GAN的方法利用从其他图像中学习到的内容来恢复高频信息,在处理新的图像时往往会产生伪影。由于,指纹图像的特征比自然图像更加复杂。因此,将以前的网络应用于指纹图像,尤其是中等分辨率的图像,会导致收敛不稳定伪影效果更加严重。针对以上弊端,本文提出了一种Enlighten-GAN超分辨率方法,来解决指纹图像的重建问题。具体来说,我们设计了启发块来控制网络收敛到一个可靠的点,并利用自我监督分层感知损失以改进损失函数提升网络性能。实验结果证明Enlighten-GAN方法在指纹图像的重建效果性能上具有更加卓越的效果。     

基于信息量制约的遥感图像融合算法

为了改善遥感图像的融合质量,制定了信息量制约法则用于获取融合遥感图像。利用亮度－色调－饱和度(IHS)变换从多光谱(MS)图像中获取亮度(I)分量。基于非下采样剪切波变换(NSST),获取I分量与全色(PAN)图像完成分解,得到I分量与PAN图像的高低频系数。最后,利用低频系数的均值以及区域能量信息,建立信息量制约法则,分析不同低频系数对融合系数的影响程度,设计不同的融合方法对这些低频系数进行融合。采用改进的平均梯度度量模型完成高频系数融合。通过NSST逆变换对计算融合后的高低频系数,输出新的亮度分量,将其与原始的色调(H)、饱和度(S)分量组合,通过IHS逆变换,获取融合结果。实验数据显示,相对已有的融合算法,所提算法的融合图像所含的信息量更为丰富,且光谱扭曲度更小。     

高光谱异常检测中背景抑制方法研究

针对高光谱图像复杂背景导致异常检测效果下降 的问题,提出了一种新的异常检 测方法。首先使用小波分解将原始高光谱图像分解成高频信息图像和低频信息图像,使用主 成分分析(PCA)方法抑制高光谱原始图像的背景信息;然后将背景抑制后图像和高频信息图 像融合,得到处理后图像;最后使用Kerner-Reed-Xiaoli(KRX)算法进行异常检测,并 仿真证明了本文方法在提高异常检测效果和效率方面的有效性。