Application of the stochastic mixing model to hyperspectral resolution enhancement

A maximum a posteriori (MAP) estimation method is described for enhancing the spatial resolution of a hyperspectral image using a higher resolution coincident panchromatic image. The approach makes use of a stochastic mixing model (SMM) of the underlying spectral scene content to develop a cost function that simultaneously optimizes the estimated hyperspectral scene relative to the observed hyperspectral and panchromatic imagery, as well as the local statistics of the spectral mixing model. The incorporation of the stochastic mixing model is found to be the key ingredient for reconstructing subpixel spectral information in that it provides the necessary constraints that lead to a well-conditioned linear system of equations for the high-resolution hyperspectral image estimate. Here, the mathematical formulation of the proposed MAP method is described. Also, enhancement results using various hyperspectral image datasets are provided. In general, it is found that the MAP/SMM method is able to reconstruct subpixel information in several principal components of the high-resolution hyperspectral image estimate, while the enhancement for conventional methods, like those based on least squares estimation, is limited primarily to the first principal component (i.e., the intensity component).     

A super-resolution reconstruction algorithm for hyperspectral images

The spatial resolution of a hyperspectral image is often coarse because of the limitations of the imaging hardware. Super-resolution reconstruction (SRR) is a promising signal post-processing technique for hyperspectral image resolution enhancement. This paper proposes a maximum a posteriori (MAP) based multi-frame super-resolution algorithm for hyperspectral images. Principal component analysis (PCA) is utilized in both parts of the proposed algorithm: motion estimation and image reconstruction. A simultaneous motion estimation method with the first few principal components, which contain most of the information of a hyperspectral image, is proposed to reduce computational load and improve motion field accuracy. In the image reconstruction part, different image resolution enhancement techniques are applied to different groups of components, to reduce computational load and simultaneously remove noise. The proposed algorithm is tested on both synthetic images and real image sequences. The experimental results and comparative analyses verify the effectiveness of this algorithm.     

Hyperspectral image super-resolution combining with deep learning and spectral unmixing

In recent years, hyperspectral image super-resolution has attracted the attention of many researchers and has become a hot topic in the field of computer vision. However, it is difficult to obtain high-resolution images due to imaging hardware devices. At present, many existing hyperspectral image super-resolution methods have not achieved good results. In this paper, we propose a hyperspectral image super-resolution method combining with deep residual convolutional neural network (DRCNN) and spectral unmixing. Firstly, the spatial resolution of the image is enhanced by learning a priori knowledge of natural images. The DRCNN reconstructs high spatial resolution hyperspectral images by concatenating multiple residual blocks, each containing two convolutional layers. Secondly, the spectral features of low-resolution and high-resolution hyperspectral images are linked by spectral unmixing. This approach aims to obtain the endmember matrix and the abundance matrix. The final reconstruction result is obtained by multiplying the endmember matrix and the abundance matrix. In addition, in order to improve the visual effect of the reconstructed image, the total variation regularity is used to impose constraints on the abundance matrix to enhance the relationship between the pixels. The experimental results of remote sensing data based on ground facts show that the proposed method has good performance and preserves spatial information and spectral information without the need for auxiliary images.     

MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor

This paper presents a novel maximum a posteriori estimator for enhancing the spatial resolution of an image using co-registered high spatial-resolution imagery from an auxiliary sensor. Here, we focus on the use of high-resolution panchomatic data to enhance hyperspectral imagery. However, the estimation framework developed allows for any number of spectral bands in the primary and auxiliary image. The proposed technique is suitable for applications where some correlation, either localized or global, exists between the auxiliary image and the image being enhanced. To exploit localized correlations, a spatially varying statistical model, based on vector quantization, is used. Another important aspect of the proposed algorithm is that it allows for the use of an accurate observation model relating the "true" scene with the low-resolutions observations. Experimental results with hyperspectral data derived from the airborne visible-infrared imaging spectrometer are presented to demonstrate the efficacy of the proposed estimator.     

Hyperspectral image quality based on convolutional network of multi-scale depth

Hyperspectral imagery has been widely used in military and civilian research fields such as crop yield estimation, mineral exploration, and military target detection. However, for the limited imaging equipment and the complex imaging environment of hyperspectral images, the spatial resolution of hyperspectral images is still relatively low, which limits the application of hyperspectral images. So, studying the data characteristics of hyperspectral images deeply and improving the spatial resolution of hyperspectral images is an important prerequisite for accurate interpretation and wide application of hyperspectral images. The purpose of this paper is to deal with super-resolution of the hyperspectral image quickly and accurately, and maintain the spectral characteristics of the hyperspectral image, makes the spectral separability of the substrate in the original image remains unchanged after super-resolution processing. This paper first learns the mapping relationship between the spectral difference of low-resolution hyperspectral image and the spectral difference of the corresponding high-resolution hyperspectral image based on multiple scale convolutional neural network, Thus, apply this mapping relationship to the input low-resolution hyperspectral image generally, getting the corresponding high resolution spectral difference. Constrained space by using the image of reconstructed spectral difference, this requires the low-resolution hyperspectral image generated by the reconstructed image is to be close to the input low-resolution hyperspectral image in space, so that the whole process becomes a closed circulation system where the low-resolution hyperspectral image generation of high-resolution hyperspectral images, then back to low-resolution hyperspectral images. This innovative design further enhances the super-resolution performance of the algorithm. The experimental results show that the hyperspectral image super-resolution method based on convolutional neural network improves the input image spatial information, and the super-resolution performance of the model is above 90%, which can maintain the spectral information well.     

MULTI-SPECTRAL AND HYPERSPECTRAL IMAGE FUSION USING 3-D WAVELET TRANSFORM

Image fusion is performed between one band of multi-spectral image and two bands of hyperspectral image to produce fused image with the same spatial resolution as source multi-spectral image and the same spectral resolution as source hyperspeetral image. According to the characteristics and 3-Dimensional （3-D） feature analysis of multi-spectral and hyperspectral image data volume, the new fusion approach using 3-D wavelet based method is proposed. This approach is composed of four major procedures： Spatial and spectral resampling, 3-D wavelet transform, wavelet coefficient integration and 3-D inverse wavelet transform. Especially, a novel method, Ratio Image Based Spectral Resampling （RIBSR）method, is proposed to accomplish data resampling in spectral domain by utilizing the property of ratio image. And a new fusion rule, Average and Substitution （A＆S） rule, is employed as the fusion rule to accomplish wavelet coefficient integration. Experimental results illustrate that the fusion approach using 3-D wavelet transform can utilize both spatial and spectral characteristics of source images more adequately and produce fused image with higher quality and fewer artifacts than fusion approach using 2-D wavelet transform. It is also revealed that RIBSR method is capable of interpolating the missing data more effectively and correctly, and A＆S rule can integrate coefficients of source images in 3-D wavelet domain to preserve both spatial and spectral features of source images more properly.     

基于sc-NMF的高光谱图像融合

将高光谱图像与全色图像融合,所得融合数据对于后续的其它高光谱图像处理非常有帮助。区别于传统方法,针对高光谱图像特点,引入了光谱约束项,改进并建立基于光谱约束的非负矩阵分解（spectral-constrained nonnegative matrix factorization,sc-NMF）。改进后,该模型首先在光谱约束前提下,对高光谱图像进行非负矩阵分解,对分解所得基底进行增强,再重建高光谱图像。这样,所得到的融合图像在空间细节和光谱保持性均有比较好的效果。最后,进行了仿真和实际数据的实验验证,通过主观和客观的评价结果,所改进的融合方法性能较好,比传统方法更适用于高光谱图像融合。     

Multi-resolution analysis techniques and nonlinear PCA for hybrid pansharpening applications

Hyperspectral images have a higher spectral resolution (i.e., a larger number of bands covering the electromagnetic spectrum), but a lower spatial resolution with respect to multispectral or panchromatic acquisitions. For increasing the capabilities of the data in terms of utilization and interpretation, hyperspectral images having both high spectral and spatial resolution are desired. This can be achieved by combining the hyperspectral image with a high spatial resolution panchromatic image. These techniques are generally known as pansharpening and can be divided into component substitution (CS) and multi-resolution analysis (MRA) based methods. In general, the CS methods result in fused images having high spatial quality but the fused images suffer from spectral distortions. On the other hand, images obtained using MRA techniques are not as sharp as CS methods but they are spectrally consistent. Both substitution and filtering approaches are considered adequate when applied to multispectral and PAN images, but have many drawbacks when the low-resolution image is a hyperspectral image. Thus, one of the main challenges in hyperspectral pansharpening is to improve the spatial resolution while preserving as much as possible of the original spectral information. An effective solution to these problems has been found in the use of hybrid approaches, combining the better spatial information of CS and the more accurate spectral information of MRA techniques. In general, in a hybrid approach a CS technique is used to project the original data into a low dimensionality space. Thus, the PAN image is fused with one or more features by means of MRA approach. Finally the inverse projection is used to obtain the enhanced image in the original data space. These methods, permit to effectively enhance the spatial resolution of the hyperspectral image without relevant spectral distortions and on the same time to reduce the computational load of the entire process. In particular, in this paper we focus our attention on the use of Nonlinear Principal Component Analysis (NLPCA) for the projection of the image into a low dimensionality feature space. However, if on one hand the NLPCA has been proved to better represent the intrinsic information of hyperspectral images in the feature space, on the other hand an analysis of the impact of different fusion techniques applied to the nonlinear principal components in order to define the optimal framework for the hybrid pansharpening has not been carried out yet. More in particular, in this paper we analyze the overall impact of several widely used MRA pansharpening algorithms applied in the nonlinear feature space. The results obtained on both synthetic and real data demonstrate that an accurate selection of the pansharpening method can lead to an effective improvement of the enhanced hyperspectral image in terms of spectral quality and spatial consistency, as well as a strong reduction in the computational time.     

基于预测与JPEG2000的高光谱图像无损压缩方法

随着成像光谱仪向着高光谱分辨率、高空间分辨率方向发展,高光谱图像的数据量呈几何级数增长。由于数据传输和存储能力的限制,必须对高光谱图像进行有效压缩。首先,对高光谱图像的相关性进行了深入分析,得知其具有一定的空间相关性和极强的谱间相关性,从而具有较强的可压缩性。其次,结合JPEG2000对DPCM进行了修改,提出了基于一阶线性预测与JPEG2000相结合的无损压缩方案。最后,在软件平台上实现了该方案,并取得了较好的压缩效果。结果表明,该方案可以有效的实现高光谱图像无损压缩,验证了方案的可行性,为硬件平台上实现该方案提供了理论依据。     

Super-resolution reconstruction of hyperspectral images.

Hyperspectral images are used for aerial and space imagery applications, including target detection, tracking, agricultural, and natural resource exploration. Unfortunately, atmospheric scattering, secondary illumination, changing viewing angles, and sensor noise degrade the quality of these images. Improving their resolution has a high payoff, but applying super-resolution techniques separately to every spectral band is problematic for two main reasons. First, the number of spectral bands can be in the hundreds, which increases the computational load excessively. Second, considering the bands separately does not make use of the information that is present across them. Furthermore, separate band super-resolution does not make use of the inherent low dimensionality of the spectral data, which can effectively be used to improve the robustness against noise. In this paper, we introduce a novel super-resolution method for hyperspectral images. An integral part of our work is to model the hyperspectral image acquisition process. We propose a model that enables us to represent the hyperspectral observations from different wavelengths as weighted linear combinations of a small number of basis image planes. Then, a method for applying super resolution to hyperspectral images using this model is presented. The method fuses information from multiple observations and spectral bands to improve spatial resolution and reconstruct the spectrum of the observed scene as a combination of a small number of spectral basis functions.     

Spectral linear mixing model in low spatial resolution image data

Different ways to estimate the spectral reflectance for the component classes in a mixture problem have been proposed in the literature (pure pixels, spectral library, field measurements). One of the most common approaches consists in the use of pure pixels, i.e., pixels that are covered by a single component class. This approach presents the advantage of allowing the extraction of the components' reflectance directly from the image data. This approach, however, is generally not feasible in the case of low spatial resolution image data, due to the large ground area covered by a single pixel. In this paper, a methodology aiming to overcome this limitation is proposed. The proposed approach makes use of the spectral linear mixing model. In the proposed methodology, the components' proportions in image data are estimated using a medium spatial resolution image as auxiliary data. The linear mixing model is then solved for the unknown spectral reflectances. Experiments are presented, using Terra Moderate Resolution Imaging Spectroradiometer (MODIS) and Landsat Enhanced Thematic Mapper Plus, as low and medium spatial resolution image data, respectively, acquired on the same date over the Tapajos study site, Brazilian Amazon. Three component classes or endmembers are present in the scene covered by the experiment, namely vegetation, exposed soil, and shade. The components' spectral reflectance for the Terra MODIS spectral bands were then estimated by applying the proposed methodology. The reliability of these estimates is appraised by analyzing scatter diagrams produced by the Terra MODIS spectral bands and also by comparing the fraction images produced using both image datasets. This methodology appears appropriate for up-scaling information for regional and global studies.     

3维卷积递归神经网络的高光谱图像分类方法

为了针对高光谱图像中空间信息与光谱信息的不同特性进行特征提取，提出一种3维卷积递归神经网络(3-D-CRNN)的高光谱图像分类方法。首先采用3维卷积神经网络提取目标像元的局部空间特征信息，然后利用双向循环神经网络对融合了局部空间信息的光谱数据进行训练，提取空谱联合特征，最后使用Softmax损失函数训练分类器实现分类。3-D-CRNN模型无需对高光谱图像进行复杂的预处理和后处理，可以实现端到端的训练，并且能够充分提取空间与光谱数据中的语义信息。结果表明，与其它基于深度学习的分类方法相比，本文中的方法在Pavia University与Indian Pines数据集上分别取得了99.94%和98.81%的总体分类精度，有效地提高了高光谱图像的分类精度与分类效果。该方法对高光谱图像的特征提取具有一定的启发意义。     

基于特征融合方法的高光谱图像分类综述

高光谱图像中包含丰富的光谱特征和空间特征,这对地表物质的分类至关重要。然而高光谱图像的空间分辨率相对较低,使得图像中存在大量的混合像素,这严重制约物质分类的精度。受到观测噪声、目标区域大小及端元易变性等因素的影响,使得高光谱图像的分类仍然面临诸多挑战。随着人工智能和信息处理技术的不断进步,高光谱图像分类已成为遥感领域的一个热点问题。首先对基于特征融合的高光谱图像分类文献进行系统综述,并对几种分类策略进行分析与比较,然后介绍高光谱图像分类的发展现状及面临的相应问题,最后提出一些可以提高分类性能的策略,从而为课题的技术研究提供指导和帮助。     

基于空谱注意力机制及预激活残差网络的高光谱图像分类算法

面向高光谱图像分类的许多深度学习算法中,由于提取的空谱特征表示鉴别性不足,其模型的分类性能有待提高。针对该问题,本文提出了一种基于空谱注意力机制及预激活残差网络的高光谱图像分类算法。首先,设计了基于空谱注意力机制的空谱特征提取模块,对空谱特征进行重校准,为空谱特征在后续联合学习时能专注于更具辨别力的通道和空间位置提供保证;其次,设计了基于预激活残差网络的空谱特征联合学习模块,其中预激活残差网络改进了原始残差构建块的网络结构,从而能在利用注意力机制重校准的空谱特征的联合学习时捕获更具鉴别性的深层空谱特征,以提高分类器的分类性能。实验结果表明,和已有的一些高光谱图像分类算法相比,所提出的算法的分类准确率更高,表明该算法能有效地获得判别能力更强的空谱特征表示。      

基于多维度特征遥感图像分类方法的研究北大核心CSCD

为了解决传统高光谱图像分类方法精度低、计算成本高及未能充分利用空-谱信息的问题,本文提出一种基于多维度并行卷积神经网络(multidimensional parallel convolutional neural network,3D-2D-1D PCNN)的高光谱图像分类方法。首先,该算法利用不同维度卷积神经网络(convolutional neural network,CNN)提取高光谱图像信息中的空-谱特征、空间特征及光谱特征;之后,采用相同并行卷积层将组合后的空-谱特征、空间特征及光谱特征进行特征融合;最后,通过线性分类器对高光谱图像信息进行精准分类。本文所提方法不仅可以提取高光谱图像中更深层次的空间特征和光谱特征信息,同时能够将光谱图像不同维度的特征进行融合,减小计算成本。在Indian Pines、Pavia Center和Pavia University数据集上对本文算法和4种传统算法进行对比实验,结果表明,本文算法均得到最优结果,分类精度分别达到了99.210%、99.755%和99.770%。     

Comprehensive review on fusion techniques for spatial information enhancement in hyperspectral imagery

The volume of data grows with the advent of multiple types of remote sensing sensors and in order to extract the most useful information there is a need to combine the data gathered from the different sources. The widely used panchromatic and multispectral imageries in many applications offer decimetric and metric spatial resolution. However, the spectral resolution of these images is poor. Hyperspectral imaging has unique characteristics of providing very fine spectral resolution in a large number of bands with decametric spatial resolution and found to be highly useful for a wide span of application areas that requires high spectral resolution. The fusion of spectral and spatial information provides an effective way of enhancing the spatial quality of hyperspectral imagery as well as a method for preserving spectral quality. This fusion process is not a trivial task as always there has been a tradeoff between the preservation of spatial and spectral quality information as in the original sources of fusion. In this paper, a review on hyperspectral pansharpening and hyperspectral multispectral fusion based approaches has been reported. The widely adopted quantitative and qualitative performance measures to verify the fusion results are highlighted. In addition, the challenges in existing fusion techniques have also been discussed.     

Information-theoretic assessment of sampled hyperspectral imagers

This work focuses on estimating the information conveyed to a user by hyperspectral image data. The goal is establishing the extent to which an increase in spectral resolution enhances the amount of usable information. Indeed, a tradeoff exists between spatial and spectral resolution due to physical constraints of multi-band sensors imaging with a prefixed SNR. After describing an original method developed for the automatic estimation of variance and correlation of the noise introduced by hyperspectral imagers, lossless interband data compression is exploited to measure the useful information content of hyperspectral data. In fact, the bit rate achieved by the reversible compression process takes into account both the contribution of the “observation” noise (i.e., information regarded as statistical uncertainty, but whose relevance to a user is null) and the intrinsic information of radiance sampled and digitized through an ideally noise-free process. An entropic model of the decorrelated image source is defined and, once the parameters of the noise, assumed to be Gaussian and stationary, have been measured, such a model is inverted to yield an estimate of the information content of the noise-free source from the code rate. Results are reported and discussed on both simulated and AVIRIS data     

Integration of Spatial–Spectral Information for Resolution Enhancement in Hyperspectral Images

In this paper, a new algorithm is proposed for resolution enhancement in hyperspectral images (HSIs). The key techniques are included: spectral unmixing and superresolution mapping, by which spatial and spectral information of HSIs is substantially fused. The proposed algorithm first represents each pixel in scene as a linear combination of landcover spectra and noise. Then, a fully constrained least squares algorithm is used to obtain the proportion of each landcover in each pixel, i.e., abundance, subjecting to two constraints: nonnegativity and sum-to-one. After that, superresolution mapping is performed on high-resolution grids according to spectral unmixing abundances of each landcover and following spatial correlation of clutters. Thus, by reasonably integrating spatial and spectral information of landcovers in HSIs, the proposed algorithm realizes resolution enhancement of the HSIs based on a back-propagation neural network. The proposed algorithm is independent from the a priori information associated with original HSIs, i.e., a main merit of the algorithm. In order to evaluate the performance of the new algorithm, numerical experiments are conducted on both simulated images and real HSIs collected by the Airborne Visible/Infrared Imaging Spectrometer. The proposed algorithm is compared with the traditional method in the experiments. The experimental results prove that the proposed algorithm effectively enhances the resolution of HSIs and indicate its applicability.     

Hyperspectral Fluorescence Imaging for Mouse Skin Tumor Detection

This paper presents a hyperspectral imaging technique based on laser‐induced fluorescence for non‐invasive detection of tumorous tissue on mouse skin. Hyperspectral imaging sensors collect image data in a number of narrow, adjacent spectral bands. Such high‐resolution measurement of spectral information reveals contiguous emission spectra at each image pixel useful for the characterization of constituent materials. The hyperspectral image data used in this study are fluorescence images of mouse skin consisting of 21 spectral bands in the visible spectrum of the wavelengths ranging from 440 nm to 640 nm. Fluorescence signal is measured with the use of laser excitation at 337 nm. An acousto‐optic tunable filter (AOTF) is used to capture images at 10 nm intervals. All spectral band images are spatially registered with the reference band image at 490 nm to obtain exact pixel correspondences by compensating the spatial offsets caused by the refraction differences in AOTF at different wavelengths during the image capture procedure. The unique fluorescence spectral signatures demonstrate a good separation to differentiate malignant tumors from normal tissues for rapid detection of skin cancers without biopsy.     

Spatial/spectral endmember extraction by multidimensional morphological operations

Spectral mixture analysis provides an efficient mechanism for the interpretation and classification of remotely sensed multidimensional imagery. It aims to identify a set of reference signatures (also known as endmembers) that can be used to model the reflectance spectrum at each pixel of the original image. Thus, the modeling is carried out as a linear combination of a finite number of ground components. Although spectral mixture models have proved to be appropriate for the purpose of large hyperspectral dataset subpixel analysis, few methods are available in the literature for the extraction of appropriate endmembers in spectral unmixing. Most approaches have been designed from a spectroscopic viewpoint and, thus, tend to neglect the existing spatial correlation between pixels. This paper presents a new automated method that performs unsupervised pixel purity determination and endmember extraction from multidimensional datasets; this is achieved by using both spatial and spectral information in a combined manner. The method is based on mathematical morphology, a classic image processing technique that can be applied to the spectral domain while being able to keep its spatial characteristics. The proposed methodology is evaluated through a specifically designed framework that uses both simulated and real hyperspectral data.