Interband structure modeling for Pan-sharpening of very high-resolution multispectral images

This paper addresses the modeling of wavelet coefficients for multispectral (MS) band sharpening based on undecimated multiresolution analysis (MRA). The coarse MS bands are sharpened by injecting highpass details taken from a high-resolution panchromatic (Pan) image. Besides the MRA, crucial point is modeling the relationships between detail coefficients of a generic MS band and the Pan image at the same resolution. Once calculated at the coarser resolution, where both types of data are available, such a model shall be extrapolated to the finer resolution in order to weight the Pan details to be injected. The goal is that the merged MS images are most similar to what the MS sensor would collect if it had the same resolution as the broadband Pan imager. Three injection models embedded in an “à trous” wavelet decomposition will be described and compared on a test set of very high-resolution QuickBird MS + Pan data. Two models work on approximation and detail coefficients, respectively, and provide a certain degree of unmixing of coarse MS pixels. The third model is based on spectral fidelity of the merged image to the (resampled) original MS data, that is, no unmixing is attempted. It is much simpler than the other two because it does not require calculation of local statistics. Fusion comparisons on spatially degraded data, of which higher-resolution true MS data are available for reference, show that the former two models yield better results than the latter, in terms of both radiometric and spectral fidelity. The latter, however, yields a reliable sharpening unaffected by local artifacts, regardless of landscape complexity. When local statistics of wavelet coefficients are used, the model estimated on approximation yields slightly better but considerably stabler results than that calculated starting from bandpass details.     

PAN‐sharpening of very high resolution multispectral images using genetic algorithms

A novel image fusion method is presented, suitable for sharpening of multispectral (MS) images by means of a panchromatic (PAN) observation. The method is based on redundant multiresolution analysis (MRA); the MS bands expanded to the finer scale of the PAN band are sharpened by adding the spatial details from the MRA representation of the PAN data. As a direct, unconditioned injection of PAN details gives unsatisfactory results, a new injection model is proposed that provides the optimum injection by maximizing a global quality index of the fused product. To this aim, a real‐valued genetic algorithm (GA) has been defined and tested on Quickbird data. The optimum GA injection is driven by an index function capable of measuring different types of possible distortions in the fused images. Fusion tests are carried out on spatially degraded data to objectively compare the proposed scheme to the most promising state‐of‐the‐art image fusion methods, and on full‐resolution image data to visually assess the performance of the proposed genetic image fusion method.     

自适应对偶树复小波－Curvelet变换的遥感图像融合

Curvelet变换克服了小波变换在处理高维信号时的不足,比小波变换具有更好的方向性、较高的逼近精度和更好的稀疏表达性能。因此将Curvelet变换应用于图像融合领域,能更好地提取图像边缘特征,为融合提取更多的特征信息。利用对偶树复小波-Curvelett变换的多尺度和多方向性特征以及自适应融合规则在选取融合系数上的优势,提出了一种基于对偶树复小波-Curvelet变换的自适应遥感图像融合新算法。算法是将全色图像和多光谱图像进行对偶树复小波-Curvelet变换分解后,针对不同的频率域特点选择不同的融合规则,对低频系数选取区域能量的加权系数自适应融合规则,对高频系数特性选用了区域特征自适应的融合规则,最后通过重构得到融合图像。将其他的融合算法和所提算法进行主观和客观的对比,结果表明,基于对偶树复小波-Curvelet变换区域特征自适应的图像融合算法是一种有效可行的图像融合算法。     

基于边缘增强与光谱特性保持的Pan-sharpening融合模型

为生成兼具高光谱质量与高空间质量的融合图像,本文提出了一种新的Pan-sharpening变分融合模型.通过拟合退化后的全色（Panchromatic,Pan）波段图像与低分辨率多光谱（Multispectral,MS）波段图像间的线性关系得到各波段MS图像的权重系数,计算从Pan图像抽取的空间细节;基于全色波段图像的梯度定义加权函数,增强了图像的强梯度边缘并对因噪声而引入的虚假边缘进行了抑制,有效地保持了全色波段图像中目标的几何结构;基于MS波段传感器的调制传输函数定义低通滤波器,自适应地限制注入空间细节的数量,显著降低了融合MS图像的光谱失真;针对Pan-sharpening模型的不适定性问题,引入L1正则化能量项,保证了数值解的稳定性.采用Split Bregman数值方法求解能量泛函的最优解,提高了算法的计算效率.QuickBird、IKONOS和GeoEye-1数据集上的实验结果表明,模型的综合融合性能优于MTF-CON、AWLP、SparseFI、TVR和MTF-Variational等算法.     

基于 MTF 和变分的全色与多光谱图像融合模型

Pan-sharpening将高分辨率图像全色(Panchromatic, Pan)波段的空间细节注入多光谱(Multispectral, MS)波段, 以生成同时具有高光谱和高空间分辨率的多光谱图像. 为改善融合效果, 需要考虑多光谱和全色波段的调制传输函数(Modulation transfer function, MTF). 本文提出了一个新的基于MTF和变分的Pan-sharpening模型. 该模型的能量泛函包括两项, 第1项为细节注入项, 基于高通滤波器从Pan波段中提取细节信息并注入融合图像;第2项为光谱保真项, 基于MTF设计多孔小波的低通滤波器以保持MS波段的多光谱信息. 在QuickBird、IKONOS和GeoEye数据集上的融合结果表明, 该模型可以生成同时具有高空间和高光谱质量的融合图像, 融合效果优于AWLP、IHS_BT、HPM-CC-PSF、NAWL、快速变分等算法.     

基于小波变换的多源图像数据融合与边缘检测方法

提出基于小波变换的多源图像数据融合和边缘检测的方法,对多源图像进行分解,将高频区域中的绝对值较大的系数作为重要小波系数;在低频区域,对逼近系数进行加权平均得到新的逼近系数,然后进行小波重构实现多源图像数据融合。应用小波变换对融合图像进行多尺度边缘检测,获取多源图像边缘,或对多源图像进行小波多尺度边缘检测,然后融合边缘。     

Multi spectral image fusion by deep convolutional neural network and new spectral loss function

In this paper we propose a new multispectral image fusion architecture. The proposed method includes two steps related to two neural networks. First the extracted spatial information, from panchromatic (Pan) image, is injected to upsampled multi-spectral (MS) image. In this step, the method employed a deep convolution neural network (DCNN) to estimate the spatial information of the MS image, according to multi-resolution analysis (MRA) scheme. This DCNN is trained by the low-spatial resolution version of Pan as an input, and by the spatial information as the target. This trained DCNN is called ‘Fusion network (FN)’. The FN, adaptively, estimates the spatial information of the MS images, and operates as an injection gain in the MRA scheme. In the second step, the spectral compensation is performed on the fused MS image. For this purpose, we used a novel loss function for this DCNN, to reduce the spectral distortion in the fused images, and simultaneously maintain the spatial information. This network is called ‘Spectral compensation network (SCN)’. Finally, the proposed method is compared to the several state-of-the-art methods on three datasets, using both full-reference and reduced reference criterion. The experimental results show that the proposed method can achieve competitive performance in both spatial and spectral information.     

Multispectral and multiresolution image fusion using particle swarm optimization

Multispectral and multiresolution image fusion is important for many multimedia and remote sensing applications, such as video surveillance, medical imaging, and satellite imaging. For the commercial satellite ??IKONOS,?? spatial resolutions of high-resolution panchromatic (PAN) and low-resolution multispectral (MS) satellite images are 1?m and 4?m, respectively. To cope with color distortion and blocking artifacts in fused images, in this study, a multispectral and multiresolution image fusion approach using PSO is proposed. The pixels of fused images in the training set are classified into several categories based on the characteristics of low-resolution MS images. Then, the smooth parameters of spatial and spectral responses between the high-resolution PAN and low-resolution MS images are determined by PSO. All the pixels within each category are normalized by its own smooth parameter so that color distortion and blocking artifacts can be greatly reduced. Based on the experimental results obtained in this study, the overall visual quality of the fused images by the proposed approach is better than that by three comparison approaches, whereas the correlation coefficients, ?? _PAN , for the fused images by the proposed approach are greater than that by three comparison approaches.     

Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients

For the quality of the fused outcome is determined by the amount of the information captured from the source images, thus, a multi-modal medical image fusion method is developed in the shift-invariant shearlet transform (SIST) domain. The two-state Hidden Markov Tree (HMT) model is extended into the SIST domain to describe the dependent relationships of the SIST coefficients of the cross-scale and inter-subbands. Base on the model, we explain why the conventional Average–Maximum fusion scheme is not the best rule for medical image fusion, and therefore a new scheme is developed, where the probability density function and standard deviation of the SIST coefficients are employed to calculate the fused coefficients. Finally, the fused image is obtained by directly applying the inverse SIST. Integrating the SIST and the HMT model, more spatial feature information of the singularities and more functional information contents can be preserved and transferred into the fused results. Visual and statistical analyses demonstrate that the fusion quality can be significantly improved over that of five typical methods in terms of entropy and mutual information, edge information, standard deviation, peak signal to noise and structural similarity. Besides, color distortion can be suppressed to a great extent, providing a better visual sense.     

非下采样Contourlet变换的图像融合及评价*

分析和研究了非下采样Contourlet图像表示方法及其在图像变换中的优点,提出一种基于非下采样Contourlet变换的图像融合方法。首先将待融合源图像分解成不同尺度、多方向的频带;然后采取不同的融合方法对分解的高低频分量进行融合处理,低频系数采取局部能量优先的加权法融合,高频系数则采取局部梯度优先的加权法融合;最后将融合的各频带进行逆非下采样Contourlet变换得到融合图像。实验表明,在几种不同的客观评价标准下,该方法优于传统的小波域中的融合效果,能有效消除小波变换带来的光谱扭曲和假边缘现象。     

基于Contourlet变换的多波段SAR图像融合

针对不同波段SAR图像的融合，该文提出了一种在Contourlet变换域融合的方法，利用Contourlet变换的充分表示图像边缘信息的能力，将图像分解为低通系数和不同方向的高频系数，对方向高频系数定义一个边缘信息量测指标，选择量测指标大的系数作为融合系数，解决了小波变换融合中图像边缘信息容易丢失的问题。通过对两波段SAR图像进行融合实验并与小波变换融合结果比较，在视觉特性与统计因子客观评价上均取得了更好的效果。     

Pan-sharpening using a guided filter

Pan-sharpening aims to integrate the spatial details of a high-resolution panchromatic (Pan) image with the spectral information of low-resolution multispectral (MS) images to produce high-resolution MS images. The key is to appropriately estimate the missing spatial details of the MS images while preserving their spectral contents. However, many existing methods extract the spatial details from the Pan image without fully considering the structures of the MS images, resulting in spectral distortion due to redundant detail injection. A guided filter can transfer the structures of the MS images into the intensity component or the low-pass approximation of the Pan image. Using the guided filter, we propose two novel pan-sharpening methods to reduce the redundant details among the MS and Pan images. Specifically, we extract the missing spatial details of the MS images by minimizing the difference between the Pan image and its corresponding filtering output, with the help of the MS images. Two different ways of using the MS images as guided images lead to two proposed methods, which can be grouped into component substitution (CS) family. Extensive experimental results over three data sets collected by different satellite sensors demonstrate the effectiveness of the proposed methods.     

Detection and validation of the body edge in low count emission tomography images

Segmentation of the body edge in tomographic images with low count, noisy edges is needed for both PET and SPECT respiratory motion correction and brain SPECT attenuation correction. To reduce noise we re-projected tomographic images and searched for edges in the projection count profiles or their spatial derivatives. Edge location versus projection angle was fitted with cosine basis functions after rejecting outliers and including information about edges of previous sections. We processed 10 s duration FDG PET of the thorax, HMPAO brain, DAT brain and lung perfusion SPECT. Stable edges for all four types of scan were achieved but with different profiles. Edges were validated against edges of coregistered CT or MRI. The best root mean square (rms) accuracy was 8.2 mm (PET) and 5.2 mm (brain SPECT). Inter-scan variability (standard deviation) in the estimated-to-control edge distance for 17 PET scans was 0.4 mm, for 25 ordered subset expectation maximisation (OSEM) reconstructed brain SPECT 1.0 mm and for 18 filtered back-projection (FBP) reconstructed brain SPECT 1.4 mm.     

基于曲波活性测度的SAR与多光谱图像融合

曲波变换是一种更适合于图像处理的多尺度几何分析方法,具有很强的方向性。结合活性测度将其应用于合成孔径雷达(SAR)图像和多光谱(MS)图像融合可以更好地表示图像中的有用特征。首先,对多光谱图像和SAR图像的R、G、B三波段分别进行曲波变换,粗尺度系数采用3×3窗口系数活性测度进行融合,细节尺度直接取大,对粗尺度和细节尺度系数重构后得到最终融合结果。采用熵、平均梯度、信噪比和扭曲程度对融合结果进行评价。实验结果表明,基于曲波活性测度的融合方法在保持MS光谱信息和提高空间分辨率上都有较好的结果。     

Identifying plant species using mid-wave infrared (2.5–6 μm) and thermal infrared (8–14 μm) emissivity spectra

Plant species discrimination using remote sensing is generally limited by the similarity of their reflectance spectra in the visible, NIR and SWIR domains. Laboratory measured emissivity spectra in the mid infrared (MIR; 2.5 μm–6 μm) and the thermal infrared (TIR; 8 μm–14 μm) domain of different plant species, however, reveal significant differences. It is anticipated that with the advances in airborne and space borne hyperspectral thermal sensors, differentiation between plant species may improve. The laboratory emissivity spectra of thirteen common broad leaved species, comprising 3024 spectral bands in the MIR and TIR, were analyzed. For each wavelength the differences between the species were tested for significance using the one way analysis of variance (ANOVA) with the post-hoc Tukey HSD test. The emissivity spectra of the analyzed species were found to be statistically different at various wavebands. Subsequently, six spectral bands were selected (based on the histogram of separable pairs of species for each waveband) to quantify the separability between each species pair based on the Jefferies Matusita (JM) distance. Out of 78 combinations, 76 pairs had a significantly different JM distance. This means that careful selection of hyperspectral bands in the MIR and TIR (2.5 μm–14 μm) results in reliable species discrimination.     

基于小波包和边缘特征的遥感图像融合算法

为了获得更多的图像细节信息,提出了一种基于边缘特征的遥感图像融合算法。该方法以小波包变换为基础,根据3个子方向的小波系数变化和边缘特征,确定融合规则。对TM多光谱图像和SPOT高分辨率图像进行融合实验,并从主观和客观两方面对融合结果进行了比较分析。仿真实验结果表明,该方法在空间细节信息的表现能力和光谱信息的保持方面都有一定程度的提高。     

遥感图像融合的非采样Contourlet变换方法

有效地融合高分辨率全色遥感图像(PAN)和低分辨率多光谱图像(MS),均衡融合结果中的空间细节信息和光谱信息两项特征指标,是多源遥感图像融合技术的难点。为了提高融合后图像的质量,提出了一种基于非采样Contourlet变换(NSCT)的融合方法。由于"非采样Contourlet变换"采用非采样滤波器组实现,具有移不变、高方向性和各向异性的特点,能够较好地弥补"采样的Contourlet变换(CT)"的缺陷,并且解决了小波变换方向性差的问题。实验中,以Landsat TM5图像和SPOT图像进行了算法的验证,并针对传统的直接替换、绝对值选大和局部方差选大等多分辨率融合算法与离散小波变换(DWT)及"采样的Contourlet变换"进行了对比分析,结果表明,本文方法在提高空间信息的同时,可以较好地保持原始多光谱图像中的光谱信息,弥补了"采样的Contourlet变换"在遥感图像融合应用中会导致严重的色彩畸变的缺陷。从而证明了NSCT在遥感图像融合领域是一种有效的多分辨率分解策略,可以被成功的应用到遥感图像融合应用中。     

A new metric based on extended spatial frequency and its application to DWT based fusion algorithms

A new quantitative metric is proposed to objectively evaluate the quality of fused imagery. The measured value of the proposed metric is used as feedback to a fusion algorithm such that the image quality of the fused image can potentially be improved. This new metric, called the ratio of spatial frequency error (rSFe), is derived from the definition of a previous measure termed “spatial frequency” (SF) that reflects local intensity variation. In this work, (1) the concept of SF is first extended by adding two diagonal SFs, then, (2) a reference SF (SF_R) is computed from the input images, and finally, (3) the error SF (SF_E) (subtracting the fusion SF from the reference SF), or the ratio of SF error (rSFe = SF_E/SF_R), is used as a fusion quality metric. The rSFe (which can be positive or negative) indicates the direction of fusion error—over-fused (if rSFe > 0) or under-fused (if rSFe < 0). Thus, the rSFe value can be back propagated to the fusion algorithm (BP fusion), thereby directing further parameter adjustments in order to achieve a better-fused image. The accuracy of the rSFe is verified with other quantitative measurements such as the root mean square error (RMSE) and the image quality index (IQI), as well as with a qualitative perceptual evaluation based on a standard psychophysical paradigm. An advanced wavelet transform (aDWT) method that incorporates principal component analysis (PCA) and morphological processing into a regular DWT fusion algorithm is implemented with two adjustable parameters—the number of levels of DWT decompositions and the length of the selected wavelet. Results with aDWT were compared to those with a regular DWT and with a Laplacian pyramid. After analyzing several inhomogeneous image groups, experimental results showed that the proposed metric, rSFe, is consistent with RMSE and IQI, and is especially powerful and efficient for realizing the iterative BP fusion in order to achieve a better image quality. Human perceptual assessment was measured and found to strongly support the assertion that the aDWT offers a significant improvement over the DWT and pyramid methods.     

MTF driven adaptive multiscale bilateral filtering for pansharpening

The recently proposed Bilateral Filter Luminance Proportional (BFLP) method extracts the high-frequency details from panchromatic (Pan) image via a multiscale bilateral filter and adds them proportionally to the multispectral (MS) image. Although this approach seems similar to other multiresolution (MRA) based schemes such as Additive Wavelet proportional Luminance (AWLP) or Generalized Laplacian (GLP) methods, multiscale bilateral filter obtains the detail planes to be injected to MS image by the combination of two Gaussian kernels controlling the transfer of details and performing successively in spatial and range domains, thus it has two parameters to be defined, namely spatial and range parameters. Since the parameter determination step considerably affects the efficiency of the method, in this paper we propose a single parameter bilateral filter by approximating the Gaussian kernel with the bicubic kernel of à trous wavelet transform (ATWT) or modulation transfer function (MTF). Moreover, we adopt an adaptive injection scheme where the range parameter is determined adaptively so as to follow the statistics of the images to be fused. The pansharpening results are compared with ATWT-based methods, as well as some state-of-the-art methods and BFLP. The visual and quantitative comparisons for Système Pour l’Observation de la Terre 7 (SPOT 7) and Pléiades 1A images, field studies supported with UAV (Unmanned Aerial Vehicle) images and digitization results of the chosen areas in Istanbul Technical University (ITU) Maslak campus confirm the superiority of the proposed detail injection approach.