Spectral change directions of multispectral subpixels in image fusion

In an image fusion process, the spatial detail in a panchromatic pixel is injected into the corresponding n-band multispectral (MS) subpixel to yield a synthetic pixel. The synthetic pixel can be regarded as the sum of three terms: the MS subpixel, a shift term, and a product of the spatial detail and an n-dimensional (n-D) spectral change vector. In this paper, the spectral change vector directions in some current image fusion methods are characterized and classified. As the spectral development of subpixels with the improvements in spatial resolution in imaging is analogous to the spectral change of MS subpixels with the improvements in spatial resolution in image fusion, the former is used as a reference to examine the spectral change vector directions in the current image fusion methods. Moreover, image haze and unmixing of mixed MS subpixels are highlighted as two aspects needing attention for further improvement in fusion quality.     

Subpixel land cover mapping based on a new spatial attraction model with spatial-spectral information

Subpixel mapping (SPM) is a technique for handling mixed pixels to derive hard classification map at finer spatial resolution. Due to simplicity and explicit physical meanings, spatial attraction model (SAM) has been proven to be an effective SPM method. SAM methods mostly differ in the way in which they compute the spatial attraction, for example, subpixel/pixel spatial attraction model (SPSAM), subpixel/subpixel spatial attraction model (MSPSAM), and the mixed spatial attraction model (MSAM). However, these methods are difficult to fully utilize the spatial-spectral information of the original remote sensing image due to diversity of the land cover classes and the limitation of the resolution of the sensor. To solve this problem, spatial attraction model with spatial-spectral information (SAM-SS) is proposed here. SAM-SS can pick up more spatial-spectral information of the original image by adding a new processing path, namely interpolation followed by spectral unmixing. Based on visual comparison and quantitative accuracy assessment, the SAM-SS shows the best performance in the four experimental results.     

Linear spectral unmixing using endmember coexistence rules and spatial correlation

Mixed pixels are often formed when surface materials are smaller than the spatial resolution of a sensor, or two or more ground features fall within a pixel. Spectral unmixing, decomposing a mixed pixel into a set of endmembers and their corresponding abundance fractions, is an important method for extracting the underlying spectral and spatial information from remote sensing images. Recent studies have shown that it is difficult to increase the accuracy of unmixing using single pixel processing. Here, we suggest combining information on the fundamental interrelations of ground components and a priori knowledge on how ground components co-exist or exclude each other according to general geographic and geomorphic relations with spectral information may allow improved unmixing. Therefore, we propose a novel spectral unmixing method to estimate endmember abundances based on linear spectral mixing model with endmember coexistence rules and spatial correlation (LSMM-R&C). This method was implemented by incorporating endmember coexistence rules along with spatial correlation into a weighted least square method. Experiments with both synthetic and real satellite images were carried out to verify the proposed method, and its performance was also evaluated in comparison to the commonly used LSMM (linear spectral mixture method), LAU (local adaptive unmixing), ISU (iterative spectral unmixing) and ISMA (iterative spectral mixture analysis) methods. LSMM-R&C showed the smallest error, and was more effective at revealing the detailed spatial distribution of endmembers’ abundance, showing high potential for solving the problem of spatial heterogeneity among neighbouring pixels.     

IKONOS image fusion process using steepest descent method with bi-linear interpolation

There are many image fusion processes to produce a high-resolution multispectral (MS) image from low-resolution MS and high-resolution panchromatic (PAN) images. But the most significant problems are colour distortion and fusion quality. Previously, we reported a fusion process that produced a1 m resolution IKONOS fused image with minimal spectral distortion. However, block distortion appeared at the edge of the curved sections of the fused image, which was reduced by performing the wavelet transformation as a post-process. Here, we propose an image fusion process using the steepest descent method with bi-linear interpolation, which can remove block distortion without using wavelet transformation. Bi-linear interpolation provides the proper initial values of the fused image, and then the steepest descent method produces the optimum results of the fusion process. These results achieve improvement on the spectral as well as spatial quality of a1 m resolution fused image when compared with other existing methods and remove block distortion completely.     

On Endmember Identification in Hyperspectral Images Without Pure Pixels: A Comparison of Algorithms

Hyperspectral imaging is an active area of research in Earth and planetary observation. One of the most important techniques for analyzing hyperspectral images is spectral unmixing, in which mixed pixels (resulting from insufficient spatial resolution of the imaging sensor) are decomposed into a collection of spectrally pure constituent spectra, called endmembers weighted by their correspondent fractions, or abundances. Over the last years, several algorithms have been developed for automatic endmember extraction. Many of them assume that the images contain at least one pure spectral signature for each distinct material. However, this assumption is usually not valid due to spatial resolution, mixing phenomena, and other considerations. A?recent trend in the hyperspectral imaging community is to design endmember identification algorithms which do not assume the presence of pure pixels. Despite the proliferation of this kind of algorithms, many of which are based on minimum enclosing simplex concepts, a rigorous quantitative and comparative assessment is not yet available. In this paper, we provide a comparative analysis of endmember extraction algorithms without the pure pixel assumption. In our experiments we use synthetic hyperspectral data sets (constructed using fractals) and real hyperspectral scenes collected by NASA’s Jet Propulsion Laboratory.     

A super resolution approach for spectral unmixing of remote sensing images

It is well known that coarse spatial resolution is an important factor for the occurrence of mixed pixels in remote sensing images, and conventional approaches for spectral unmixing adopt various techniques on spectral dimension only in a fixed spatial resolution. In this article, a super resolution (SR) approach for spectral unmixing is proposed, based on the assumption that increasing the spatial resolution helps to retrieve the composition of a pixel. Firstly, a remote sensing image is downscaled into an SR image using example-based kernel ridge regression (EBKRR). Secondly, the SR image is classified using supervised hard classification, and then the class map is decomposed into thematic class layers. Thirdly, the thematic class layers are upscaled into the original spatial resolution with an averaging operation, and the abundance maps are finally derived. In two simulated data-based experiments and one ground data-based experiment, this approach was compared with linear spectral mixture analysis (LSMA) and artificial neural network (ANN)-based spectral unmixing methods. The accuracy assessment indicated that the SR approach outperformed LSMA and ANN under measurements of mean absolute error and absolute bias in the three experiments.     

形态学滤波和改进PCNN的NSST域多光谱与全色图像融合

目的 全色图像的空间细节信息增强和多光谱图像的光谱信息保持通常是相互矛盾的,如何能够在这对矛盾中实现最佳融合效果一直以来都是遥感图像融合领域的研究热点与难点。为了有效结合光谱信息与空间细节信息,进一步改善多光谱与全色图像的融合质量,提出一种形态学滤波和改进脉冲耦合神经网络（PCNN）的非下采样剪切波变换（NSST）域多光谱与全色图像融合方法。方法 该方法首先分别对多光谱和全色图像进行非下采样剪切波变换;对二者的低频分量采用形态学滤波和高通调制框架（HPM）进行融合,将全色图像低频子带的细节信息注入到多光谱图像低频子带中得到融合后的低频子带;对二者的高频分量则采用改进脉冲耦合神经网络的方法进行融合,进一步增强融合图像中的空间细节信息;最后通过NSST逆变换得到融合图像。结果 仿真实验表明,本文方法得到的融合图像细节信息清晰且光谱保真度高,视觉效果上优势明显,且各项评价指标与其他方法相比整体上较优。相比于5种方法中3组融合结果各指标平均值中的最优值,清晰度和空间频率分别比NSCT-PCNN方法提高0.5%和1.0%,光谱扭曲度比NSST-PCNN方法降低4.2%,相关系数比NSST-PCNN方法提高1.4%,信息熵仅比NSST-PCNN方法低0.08%。相关系数和光谱扭曲度两项指标的评价结果表明本文方法相比于其他5种方法能够更好地保持光谱信息,清晰度和空间频率两项指标的评价结果则展示了本文方法具有优于其他对比方法的空间细节注入能力,信息熵指标虽不是最优值,但与最优值非常接近。结论 分析视觉效果及各项客观评价指标可以看出,本文方法在提高融合图像空间分辨率的同时,很好地保持了光谱信息。综合来看,本文方法在主观与客观方面均具有优于亮度色调饱和度（IHS）法、主成分分析（PCA）法、基于非负矩阵分解（CNMF）、基于非下采样轮廓波变换和脉冲耦合神经网络（NSCT-PCNN）以及基于非下采样剪切波变换和脉冲耦合神经网络（NSST-PCNN）5种经典及现有流行方法的融合效果。     

Multispectral and multiresolution image fusion using particle swarm optimization

Multispectral and multiresolution image fusion is important for many multimedia and remote sensing applications, such as video surveillance, medical imaging, and satellite imaging. For the commercial satellite ??IKONOS,?? spatial resolutions of high-resolution panchromatic (PAN) and low-resolution multispectral (MS) satellite images are 1?m and 4?m, respectively. To cope with color distortion and blocking artifacts in fused images, in this study, a multispectral and multiresolution image fusion approach using PSO is proposed. The pixels of fused images in the training set are classified into several categories based on the characteristics of low-resolution MS images. Then, the smooth parameters of spatial and spectral responses between the high-resolution PAN and low-resolution MS images are determined by PSO. All the pixels within each category are normalized by its own smooth parameter so that color distortion and blocking artifacts can be greatly reduced. Based on the experimental results obtained in this study, the overall visual quality of the fused images by the proposed approach is better than that by three comparison approaches, whereas the correlation coefficients, ?? _PAN , for the fused images by the proposed approach are greater than that by three comparison approaches.     

Fusion of multispectral and panchromatic satellite sensor imagery based on tailored filtering in the Fourier domain

A new methodology for fusing satellite sensor imagery, based on tailored filtering in the Fourier domain is proposed. Finite‐duration Impulse Response (FIR) filters have been designed through an objective criterion, which depends on source image characteristics only. The designed filters allow a weighted fusion of the information contained in a fine spatial resolution image (PAN) and in a multispectral image (MULTI), respectively, establishing a trade‐off between spatial and spectral quality of the resulting fused image. This new technique has been tested with Landsat Enhanced Thematic Mapper Plus (ETM+) imagery. Spatial and spectral quality of the fused images was compared with the results provided by Mallat's Wavelet algorithm. The images fused by the proposed method were characterized by a spatial resolution very close to the PAN image, and by the spectral resolution of the MULTI image.     

Iterative spectral unmixing (ISU)

Spectral unmixing techniques strive to find proportions of end-members within a pixel from the observed mixed pixel spectrum and a number of pure end-member spectra of known composition. The outcomes of such analysis are fraction (abundance) images for the selected (pure) end-members and a root mean square (RMS) error estimate representing the difference between the observed mixed spectrum and the calculated mixed spectrum. The RMS image can be used to select additional end-members and re-position existing ones. This is now done manually. In this Letter, an automatediterative approach is proposed using the RMS error image to select additional end-members and re-distribute older ones in order toincrease the accuracy of the spectral unmixing. Optimization criteria are proposed to drive the iterative process including minimization of the average RMS, minimizing the spread of the RMS values, minimizing the spatial structure of the RMS image, minimizing the spatial anisotropy of the RMS image and minimizing the local variance. The preliminary results of the analysis indicate that considerable improvement tothe spectral unmixing results are achieved using the iterative spectral unmixing (ISU) approach.     

Two improvement schemes of PAN modulation fusion methods for spectral distortion minimization

Fusion of panchromatic (PAN) and multispectral (MS) images is one of the most promising issues in remote sensing. PAN modulation fusion methods are usually based on an assumption that a ratio of two different‐resolution versions of an MS band is equal to a ratio of two different‐resolution versions of a PAN image. In such fusion methods, image haze is rarely taken into account, and it may produce serious spectral distortion in synthetic images. In this paper, assuming that the previous ratio relationship only holds for haze‐free images, two relevant improvement schemes are proposed to better express the ratio relationship of haze‐included images. In a test on a spatially degraded IKONOS dataset, the first scheme synthesizes an image with minimum spectral distortion, and the second modifies several current PAN modulation fusion methods and generates high‐quality synthetic products. The experiment results confirm that image haze can seriously impact the quality of fused images obtained by using PAN modulation fusion methods, and it should be taken into account in relevant image fusion.     

基于光谱归一化的变组分光谱混合分析(NMESMA)方法及其应用

混合像元问题在低、中分辨率遥感图像中尤为突出,混合像元的存在不仅会影响地物识别和图像分类精度,也是遥感科学向定量化发展的主要障碍之一。因此,遥感图像混合像元分解及其地表覆盖信息的定量提取是近年来研究的热点。针对城市土地覆盖信息的定量提取问题,利用中等分辨率遥感图像(Landsat TM),集成光谱归一化与变组分光谱混合分析(NMESMA)的方法,基于植被-非渗透表面-土壤(V\|I\|S)模型,定量提取研究区植被、土壤和非渗透表面3类土地覆盖的定量信息,并与固定组分的光谱混合分析(LSMA)分解结果进行对比分析。结果表明:基于光谱归一化的变组分光谱混合分析(NMESMA)方法获得的精度高于传统固定组分的光谱混合分析(LSMA)结果,可有效解决光谱异质性较高的城市区域的混合像元问题,为有效提取城市地表覆盖信息,研究城市生态环境变化和模拟分析,提供了有效的信息提取方法。     

灰色关联分析与模糊推理边缘检测图像融合法

为解决融合图像不同程度的光谱失真问题,提出了一种结合灰色关联分析、模糊推理和IHS变换的图像融合算法。首先通过灰色关联分析和模糊推理算出全色图像的边缘点和非边缘点,得到丰富的边缘信息,然后对多光谱图像进行IHS变换,以亮度分量为依据对全色图像进行直方图匹配,再基于边缘信息对亮度分量和直方图匹配后的全色图像进行线性加权,最后通过IHS逆变换得到融合图像。为验证本文方法的有效性,与5种常用方法比较,从视觉和定量两方面进行评价,且采用降尺度评价和全分辨率评价。结果表明,该方法得到的融合图像比其他5种方法更优越。本文方法不仅提高了遥感图像的空间分辨率,也较好保留了多光谱图像的光谱信息。     

TM and IRS-1C-PAN data fusion using multiresolution decomposition methods based on the 'a tròus' algorithm

Image fusion represents an important tool for remote sensing data elaborations. This technique is used for many purposes. Very often it is used to produce improved spatial resolution. The most common situation is represented by a pair of images: the first acquired by a multispectral sensor with a pixel size greater than the pixel size of the second image given by a panchromatic sensor (PAN). Starting from these images fusion produces a new multispectral image with a spatial resolution equal, or close, to that of the PAN. Very often fusion introduces important distortions on the pixel spectra. This fact could compromise the extraction of information from the image, especially when using an automatic algorithm based on spectral signature such as in the case of image classification. In this work we present the analysis of two fusion methods based on multiresolution decomposition obtained using the 'a tròus' algorithm and applied to a pair of images acquired by Thematic Mapper (TM) and Indian Remote Sensing (IRS)-1C-PAN sensors. The methods studied are also compared with two classical fusion methods, the intensity, hue and saturation (IHS) and standardized principal components (SPC). Fused results are studied and compared using various tests including supervised classification. Most of the tests used have been extracted from literature regarding the assessment of spatial and spectral quality of fused images. This study shows that the methods based on multiresolution decomposition outperform the classical fusion methods considered with respect to spectral content preservation. Moreover, it is shown that some of the quality tests are more significant than others. The discussion of this last aspect furnishes important indications for data quality assessment methods.     

Super resolution land cover mapping of hyperspectral images using the deep image prior-based approach

ABSTRACT

Due to the instantaneous field-of-view (IFOV) of the sensor and diversity of land cover types, some pixels, usually named mixed pixels, contain more than one land cover type. Soft classification can predict the portion of each land cover type in mixed pixels in the absence of spatial distribution. The spatial distribution information in mixed pixels can be solved by super resolution mapping (SRM). Typically, SRM involves two steps: soft class value estimation, which is similar to the image super resolution of image restoration, and land cover allocation. A new SRM approach utilizes a deep image prior (DIP) strategy combined with a super resolution convolutional neural network (SRCNN) to estimate fine resolution fraction images for each land cover type; then, a simple and efficient classifier is used to allocate subpixel land cover types under the constraint of the generated fine fraction images. The proposed approach can use prior information of input images to update network parameters and no longer require training data. Experiments on three different cases demonstrate that the subpixel classification accuracy of the proposed DIP-based SRM approach is significantly better than the three conventional SRM approaches and a transfer learning-based neural network SRM approach. In addition, the DIP-SRM approach performs very robustly about small-area objects within multiple land cover types and significantly reduces soft classification uncertainty. The results of this paper provide an extension for utilizing SRCNN to address SRM issues in hyperspectral images.     

Estimating subpixel fire sizes and temperatures from ASTER using multiple endmember spectral mixture analysis

The Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) shortwave infrared subsystem can acquire images of active fires during daytime and night-time from a polar orbit, providing useful data on fire properties at a nominal spatial resolution of 30 m. Binary fire/no-fire counts of ASTER pixels have also been useful in evaluating the performance of widely-used fire products from the Moderate-Resolution Imaging Spectroradiometer (MODIS), which have a nominal spatial resolution of 1 km. However, the ASTER fire pixels are actually mixed pixels that can contain flaming, smouldering and non-burning components, and ASTER fire pixel counts provide no information about the sizes or temperatures of these subpixel components. This paper uses multiple endmember spectral mixture analysis (MESMA) to estimate subpixel fire sizes and temperatures from a night-time ASTER image of a fire in California, USA, demonstrating new methods that can provide information on fires not available from other sources. As a fire's size and its temperature exert strong influences on its gas and aerosol emissions, ecological impact and spreading rates, these MESMA estimates from ASTER imagery could contribute valuable new information towards monitoring, forecasting and understanding the behaviour and impacts of many fires worldwide.     

PAN‐sharpening of very high resolution multispectral images using genetic algorithms

A novel image fusion method is presented, suitable for sharpening of multispectral (MS) images by means of a panchromatic (PAN) observation. The method is based on redundant multiresolution analysis (MRA); the MS bands expanded to the finer scale of the PAN band are sharpened by adding the spatial details from the MRA representation of the PAN data. As a direct, unconditioned injection of PAN details gives unsatisfactory results, a new injection model is proposed that provides the optimum injection by maximizing a global quality index of the fused product. To this aim, a real‐valued genetic algorithm (GA) has been defined and tested on Quickbird data. The optimum GA injection is driven by an index function capable of measuring different types of possible distortions in the fused images. Fusion tests are carried out on spatially degraded data to objectively compare the proposed scheme to the most promising state‐of‐the‐art image fusion methods, and on full‐resolution image data to visually assess the performance of the proposed genetic image fusion method.     

Remote-sensing image fusion using sparse representation with sub-dictionaries

Remote-sensing image fusion aims to obtain a multispectral (MS) image with a high spatial resolution, which integrates spatial information from the panchromatic (Pan) image and with spectral information from the MS image. Sparse representation (SR) has been recently used in remote-sensing image fusion method, and can obtain superior results to many traditional methods. However, the main obstacle is that the dictionary is generated from high resolution MS images (HRMS), which are difficult to acquire. In this article, a new SR-based remote-sensing image fusion method with sub-dictionaries is proposed. The image fusion problem is transformed into a restoration problem under the observation model with the sparsity constraint, so the fused HRMS image can then be reconstructed by a trained dictionary. The proposed dictionary for image fusion is composed of several sub-dictionaries, each of which is constructed from a source Pan image and its corresponding MS images. Therefore, the dictionary can be constructed without other HRMS images. The fusion results from QuickBird and IKONOS remote-sensing images demonstrate that the proposed method gives higher spatial resolution and less spectral distortion compared with other widely used and the state-of-the-art remote-sensing image fusion methods.