高光谱图像超分辨率重建技术研究进展

不同于传统图像（如灰度图像、RGB图像等）专注于保存目标场景的空间信息,高光谱图像蕴含丰富的空—谱信息,不仅可以保存目标的空间信息,还可以保存具有高可辨性的光谱信息。因此高光谱图像广泛应用于多种计算机视觉和遥感图像任务中,如目标检测、场景分类和目标追踪等。然而,在高光谱图像获取以及重建过程中仍然存在许多问题与瓶颈。如传统高光谱成像仪器在成像过程中通常会引入噪声,且获得的图像往往具有较低的空间分辨率,极大地影响了高光谱图像的质量,对后续数据分析任务造成了极大的困难。近年来,高光谱图像超分辨率重建技术研究得到了极大的发展,现有超分辨率重建方法可以大致分为两类,一类为空间超分辨率重建方法,可以通过直接提升高光谱图像的空间分辨率来获得高质量高光谱图像;另一类为光谱超分辨率重建方法,可以通过提升高空间分辨率图像的光谱分辨率来生成高质量高光谱图像。本文从高光谱图像超分辨率重建领域的新设计、新方法和应用场景出发,通过综合国内外前沿文献来梳理该领域的主要发展,重点论述高光谱图像超分辨率重建领域的发展现状、前沿动态、热点问题及趋势。     

高光谱图像空谱特征提取综述

由于高光谱图像包含了丰富的光谱、空间和辐射信息，且具有光谱接近连续、图谱合一的特性，可用于地质勘探、精细农业、生态环境、城市遥感以及军事目标检测等领域的目标精准分类与识别。对高光谱图像进行空谱特征提取是遥感领域的研究热点和前沿课题之一。传统空谱特征提取方法对高光谱图像分类的计算量和样本需求小、理论可解释性好、抗噪声能力强，但应用于分类的精度受限于特征来源；基于深度学习的高光谱图像空谱特征提取方法虽然计算量和样本需求大，但是由于深层空谱特征的表达能力更好，可以大幅度提高分类器的性能。为了便于对高光谱图像空谱特征提取领域进行更深入有效的探索，本文系统综述了相关研究进展。首先，概述了空间纹理与形态学特征提取、空间邻域信息获取及空间信息后处理等传统高光谱空谱特征提取方法的原理，对大量的已有工作进行了梳理、分析与总结。然后，从深度空谱特征提取角度出发，介绍了当前流行的卷积神经网络、图卷积神经网络及跨场景多源数据模型的结构特点及研究进展，分析、评价了基于深度学习的网络模型对高光谱图像空谱特征提取的优势及问题所在。最后，对该研究领域的未来相关发展提出建议并进行了展望。     

Hybrid spatial-spectral feature in broad learning system for Hyperspectral image classification

Hyperspectral images (HSIs) classification have aroused a great deal of attention recently due to their wide range of practical prospects in numerous fields. Spatial-spectral fusion feature is widely used in HSI classification to get better performance. These methods are mostly based on a simple linear addition with the combined hyper-parameter to fuse the spatial and spectral information. It is necessary to fuse the features in a more suitable method. To solve this problem, we propose a novel HSI classification approach based on Hybrid spatial-spectral feature in broad learning system (HSFBLS). First, we employ an adaptive weighted mean filter to obtain spatial feature. Computing the weights of spatial and spectral channels in hybrid module by two BLS and uniting them with a weighted linear function. Then, we fuse the spectral-spatial feature by sparse autoencoder to get weighted fusion feature as the feature nodes to classify HSI data in BLS. By a two-stage fusion of spatial and spectral information, it can increase the classification accuracy contrast to simple combination. Very satisfactory classification results on typical HSI datasets illustrate the availability of proposed HSFBLS. Moreover, HSFBLS also reduce training time greatly contrast to time-consuming network.

     

基于焦点损失的半监督高光谱图像分类

针对高光谱图像（HSI）训练数据获取困难的问题,采用了一种新的HSI半监督分类框架,该框架利用有限的标记数据和丰富的未标记数据来训练深度神经网络。同时,由于高光谱样本分布是不平衡的,导致不同样本分类难度存在巨大差异,采用原始交叉熵损失函数无法刻画这种分布特征,因而分类效果不理想。为了解决这个问题,在半监督分类框架中提出一种基于焦点损失的多分类目标函数。最后,考虑到HSI的空间信息对分类的影响,结合马尔可夫随机场（MRF）,利用样本空间特征进一步改善分类效果。在两个常用的HSI数据集上,将所提方法与多种典型算法进行了实验对比分析,实验结果表明所提方法能够产生优于其他对比方法的分类效果。     

Spatial-spectral neighbour graph for dimensionality reduction of hyperspectral image classification

Recently, graph embedding-based methods have drawn increasing attention for dimensionality reduction (DR) of hyperspectral image (HSI) classification. Graph construction is a critical step for those DR methods. Pairwise similarity graph is generally employed to reflect the geometric structure in the original data. However, it ignores the similarity of neighbouring pixels. In order to further improve the classification performance, both spectral and spatial-contextual information should be taken into account in HSI classification. In this paper, a novel spatial-spectral neighbour graph (SSNG) is proposed for DR of HSI classification, which consists of the following four steps. First, a superpixel-based segmentation algorithm is adopted to divide HSI into many superpixels. Second, a novel distance metric is utilized to reflect the similarity of two spectral pixels in each superpixel. In the third step, a spatial-spectral neighbour graph is constructed according to the above distance metric. At last, support vector machine with a composite kernel (SVM-CK) is adopted to classify the dimensionality-reduced HSI. Experimental results on three real hyperspectral datasets demonstrate that our method can achieve higher classification accuracy with relatively less consumed time than other graph embedding-based methods.     

基于空谱融合网络的高光谱图像分类方法

针对高光谱图像分类中提取的空-谱特征表达能力弱及维数较高的问题,提出一种基于空-谱融合网络（SSF-Net）的高光谱图像分类方法。首先,利用双通道卷积神经网络（Two-CNN）同时提取高光谱图像的光谱和空间特征;其次,使用多模态压缩双线性池化（MCB）将所提取的多模态特征向量的外积投射到低维空间,以此产生空-谱联合特征。该特征融合网络,既可以分析光谱特征和空间特征向量中元素之间的复杂关系,同时也避免对光谱和空间向量直接进行外积计算,造成维数过高、计算困难的问题。最终实验表明,与现有基于神经网络的分类方法相比,所提出的高光谱图像分类算法能够获得更高的像元分类精度,表明该网络所提取的空-谱联合向量对高光谱图像具有更强的特征表达能力。     

Unsupervised spectral sub-feature learning for hyperspectral image classification

Spectral pixel classification is one of the principal techniques used in hyperspectral image (HSI) analysis. In this article, we propose an unsupervised feature learning method for classification of hyperspectral images. The proposed method learns a dictionary of sub-feature basis representations from the spectral domain, which allows effective use of the correlated spectral data. The learned dictionary is then used in encoding convolutional samples from the hyperspectral input pixels to an expanded but sparse feature space. Expanded hyperspectral feature representations enable linear separation between object classes present in an image. To evaluate the proposed method, we performed experiments on several commonly used HSI data sets acquired at different locations and by different sensors. Our experimental results show that the proposed method outperforms other pixel-wise classification methods that make use of unsupervised feature extraction approaches. Additionally, even though our approach does not use any prior knowledge, or labelled training data to learn features, it yields either advantageous, or comparable, results in terms of classification accuracy with respect to recent semi-supervised methods.     

波段自适应细节注入的高分五号与Sentinel-2遥感影像空谱融合

目的 针对当前空谱融合方法应用到高光谱图像融合时,出现的空间细节信息提升明显但光谱失真,或者光谱保真度高但空间细节信息提升不足的问题,本文提出一种波段自适应细节注入的高分五号（GF-5）高光谱图像（30 m）与Sentinel-2多光谱图像（10 m）的遥感影像空谱融合方法。方法 首先,为了解决两个多波段图像不便于直接融合的问题,提出一种波段自适应的融合策略,对多光谱图像波谱范围以外的高光谱图像波段,以相关系数为标准将待融合图像进行分组。其次,针对传统Gram-Schmidt （GS）融合方法用平均权重系数模拟低分辨率图像造成的光谱失真问题,使用最小均方误差估计计算线性拟合系数,再将拟合图像作为第1分量进行GS正变换,提升融合图像的光谱保真度。最后,为了能同时注入更多的空间细节信息,通过非下采样轮廓波变换将拟合图像、空间细节信息图像和多光谱图像的空间、光谱信息融入到重构的高空间分辨率图像中,再将其与其他GS分量一起进行逆变换,最终得到10 m分辨率的GF-5融合图像。结果 通过与当前用于高光谱图像空谱融合的典型方法比较,本文方法对于受时相影响较小的城镇区域,在提升空间分辨率的同时有较好的光谱保真度,且不会出现噪点;对于受时相变化影响大的植被密集区域,本文方法融合图像有较好的清晰度和地物细节信息,且没有噪点出现。本文方法的CC （correlation coefficient）、ERGAS （erreur relative globale adimensionnelle de synthèse）和SAM （spectral angle mapper）相比于传统GS方法分别提升8%、26%和28%,表明本文方法的光谱保真度大大提高。结论 本文方法的结果空间上没有噪点且光谱曲线与原始光谱曲线基本保持一致,是一种兼具高空间分辨率和高光谱保真度的高光谱图像融合方法。     

基于自适应最近邻的局部保持投影算法

局部保持投影（LPP）是一种新的数据降维技术,但其本身是一种非监督学习算法,对于分类问题效果不是太好。基于自适应最近邻,结合LPP算法,提出了一种有监督的局部保持投影算法（ANNLPP）。该方法通过修改LPP算法中的权值矩阵,在降维的同时,增加了类别信息,是一种有监督学习算法。通过二维数据可视化和UMIST、ORL 人脸识别实验,表明该方法对于分类问题具有较好的降维效果。     

A semi-supervised system for weed mapping in sunflower crops using unmanned aerial vehicles and a crop row detection method

This paper presents a system for weed mapping, using imagery provided by unmanned aerial vehicles (UAVs). Weed control in precision agriculture is based on the design of site-specific control treatments according to weed coverage. A key component is precise and timely weed maps, and one of the crucial steps is weed monitoring, by ground sampling or remote detection. Traditional remote platforms, such as piloted planes and satellites, are not suitable for early weed mapping, given their low spatial and temporal resolutions. Nonetheless, the ultra-high spatial resolution provided by UAVs can be an efficient alternative. The proposed method for weed mapping partitions the image and complements the spectral information with other sources of information. Apart from the well-known vegetation indexes, which are commonly used in precision agriculture, a method for crop row detection is proposed. Given that crops are always organised in rows, this kind of information simplifies the separation between weeds and crops. Finally, the system incorporates classification techniques for the characterisation of pixels as crop, soil and weed. Different machine learning paradigms are compared to identify the best performing strategies, including unsupervised, semi-supervised and supervised techniques. The experiments study the effect of the flight altitude and the sensor used. Our results show that an excellent performance is obtained using very few labelled data complemented with unlabelled data (semi-supervised approach), which motivates the use of weed maps to design site-specific weed control strategies just when farmers implement the early post-emergence weed control.     

Onboard target detection in hyperspectral image based on deep learning with FPGA implementation

Onboard target detection of Hyperspectral Imagery (HSI) is widely adopted in the field of remote sensing. It requires high detection accuracy and low computational complexity for processing a large volume of HSI data. In this study, a Locally Preserving Discriminative Broad Learning (LPDBL) was introduced for target detection due to its simple, excellent generalization ability, and its competitive performance. The detection was done through spatial-spectral information, band selection, and estimation of the covariance matrix. The fisher discriminant method was used to reduce the dimension of HSI data. Weights was adjusted through manifold regularization in order to enhance the detection ability of the proposed method. To study the performance of the proposed LPDBL, experiment was conducted on two different datasets of HSI. The results revealed that the proposed method performed better and suitable for target detection. The LPDBL was implemented on Virtex-7 Field Programmable Gate Array (FPGA) board. Furthermore, the LPDBL technique was practically validated by two different techniques such as a broad learning system (BLS) and Automatic Target Detection in HSI (ATD-HSI). The result obtained from the FPGA was very close to the actual target position.     

Noise reduction and feature extraction based on low-rank representation and pairwise constraint preserving for hyperspectral images

ABSTRACT

Feature extraction (FE) methods play a central role in the classification of hyperspectral images (HSIs). However, all traditional FE methods work in original feature space (OFS), OFS may suffer from noise, outliers and poorly discriminative features. This paper presents a feature space enriching technique to address the problems of noise, outliers and poorly discriminative features which may exist in OFS. The proposed method is based on low-rank representation (LRR) with the capability of pairwise constraint preserving (PCP) termed LRR-PCP. LRR-PCP does not change the dimension of OFS and only can be used as an appropriate preprocessing procedure for any classification algorithm or DR methods. The proposed LRR-PCP aims to enrich the OFS and obtain extracted feature space (EFS) which results in features richer than OFS. The problems of noise and outliers can be decreased using LRR. But, LRR cannot preserve the intrinsic local structure of the original data and only capture the global structure of data. Therefore, two additional penalty terms are added into the objective function of LRR to keep the local discriminative ability and also preserve the data diversity. LRR-PCP method not only can be used in supervised learning but also in unsupervised and semi-supervised learning frameworks. The effectiveness of LRR-PCP is investigated on three HSI data sets using some existing DR methods and as a denoising procedure before the classification task. All experimental results and quantitative analysis demonstrate that applying LRR-PCP on OFS improves the performance of the classification and DR methods in supervised, unsupervised, and semi-supervised conditions.     

3D卷积自编码器高光谱图像分类模型

目的 高光谱图像分类是遥感领域的基础问题，高光谱图像同时包含丰富的光谱信息和空间信息，传统模型难以充分利用两种信息之间的关联性，而以卷积神经网络为主的有监督深度学习模型需要大量标注数据，但标注数据难度大且成本高。针对现有模型的不足，本文提出了一种无监督范式下的高光谱图像空谱融合方法，建立了3D卷积自编码器（3D convolutional auto-encoder，3D-CAE）高光谱图像分类模型。方法 3D卷积自编码器由编码器、解码器和分类器构成。将高光谱数据预处理后，输入到编码器中进行无监督特征提取，得到一组特征图。编码器的网络结构为3个卷积块构成的3D卷积神经网络，卷积块中加入批归一化技术防止过拟合。解码器为逆向的编码器，将提取到的特征图重构为原始数据，用均方误差函数作为损失函数判断重构误差并使用Adam算法进行参数优化。分类器由3层全连接层组成，用于判别编码器提取到的特征。以3D-CNN （three dimensional convolutional neural network）为自编码器的主干网络可以充分利用高光谱图像的空间信息和光谱信息，做到空谱融合。以端到端的方式对模型进行训练可以省去复杂的特征工程和数据预处理，模型的鲁棒性和稳定性更强。结果 在Indian Pines、Salinas、Pavia University和Botswana等4个数据集上与7种传统单特征方法及深度学习方法进行了比较，本文方法均取得最优结果，总体分类精度分别为0.948 7、0.986 6、0.986 2和0.964 9。对比实验结果表明了空谱融合和无监督学习对于高光谱遥感图像分类的有效性。结论 本文模型充分利用了高光谱图像的光谱特征和空间特征，可以做到无监督特征提取，无需大量标注数据的同时分类精度高，是一种有效的高光谱图像分类方法。     

用于多光谱和高光谱图像融合的联合自注意力Transformer

目的 将高光谱图像和多光谱图像进行融合，可以获得具有高空间分辨率和高光谱分辨率的光谱图像，提升光谱图像的质量。现有的基于深度学习的融合方法虽然表现良好，但缺乏对多源图像特征中光谱和空间长距离依赖关系的联合探索。为有效利用图像的光谱相关性和空间相似性，提出一种联合自注意力的Transformer网络来实现多光谱和高光谱图像融合超分辨。方法 首先利用联合自注意力模块，通过光谱注意力机制提取高光谱图像的光谱相关性特征，通过空间注意力机制提取多光谱图像的空间相似性特征，将获得的联合相似性特征用于指导高光谱图像和多光谱图像的融合；随后，将得到的融合特征输入到基于滑动窗口的残差Transformer深度网络中，探索融合特征的长距离依赖信息，学习深度先验融合知识；最后，特征通过卷积层映射为高空间分辨率的高光谱图像。结果 在CAVE和Harvard光谱数据集上分别进行了不同采样倍率下的实验，实验结果表明，与对比方法相比，本文方法从定量指标和视觉效果上，都取得了更好的效果。本文方法相较于性能第二的方法EDBIN （enhanced deep blind iterative network），在CAVE数据集上峰值信噪比提高了0.5 dB，在Harvard数据集上峰值信噪比提高了0.6 dB。结论 本文方法能够更好地融合光谱信息和空间信息，显著提升高光谱融合超分图像的质量。     

多尺度超像素分割和奇异谱分析的高光谱影像分类

目的 高光谱影像（hyperspectral image,HSI）中“同物异谱,异物同谱”的现象普遍存在,使分类结果存在严重的椒盐噪声问题。HSI中的空间地物结构复杂多样,单一尺度的空间特征提取方法无法有效地表达地物类间差异和区分地物边界。有效解决光谱混淆和空间尺度问题是提高分类精度的关键。方法 结合多尺度超像素和奇异谱分析,提出一种新的高光谱影像分类方法,从而充分挖掘地物的局部空间特征和光谱特征,解决空间尺度和光谱混淆的问题,提高分类精度。利用多尺度超像素对影像进行分割,获取不同尺度的分割影像,同时在分割区域内进行均值滤波,减少类内的光谱差异,增强类间的光谱差异;对每个区域计算平均光谱向量,并利用奇异谱分析方法获取光谱的主要鉴别特征,同时消除噪声的影响;利用支持向量机对不同尺度超像素分割影像进行分类,并进行决策融合,得到最终的分类结果。结果 实验选取了两个标准高光谱数据集和一个真实数据集,结果表明,利用本文算法提取的光谱—空间特征进行分类,比直接在原始数据上进行分类分别提高约26.8%、9.2%和13%的精度;与先进的深度学习SSRN （spectral-spatial residual network）算法相比,本文算法在精度上分别提升约5.2%、0.7%和4%,并且运行时间仅为前者的18.3%、45.4%和62.1%,处理效率更高。此外,在训练样本有限的情况下,两个标准数据集的样本分别为1%和0.2%时,本文算法均能取得87%以上的分类精度。结论 针对高光谱影像分类中的难题,提出一种新的融合光谱和多尺度空间特征的HSI分类方法。实验结果表明,本文方法优于对比方法,可以产生更精细的分类结果。     

双卷积池化结构的3D-CNN高光谱遥感影像分类方法

目的 高光谱遥感影像数据包含丰富的空间和光谱信息,但由于信号的高维特性、信息冗余、多种不确定性和地表覆盖的同物异谱及同谱异物现象,导致高光谱数据结构呈高度非线性。3D-CNN（3D convolutional neural network）能够利用高光谱遥感影像数据立方体的特性,实现光谱和空间信息融合,提取影像分类中重要的有判别力的特征。为此,提出了基于双卷积池化结构的3D-CNN高光谱遥感影像分类方法。方法 双卷积池化结构包括两个卷积层、两个BN（batch normalization）层和一个池化层,既考虑到高光谱遥感影像标签数据缺乏的问题,也考虑到高光谱影像高维特性和模型深度之间的平衡问题,模型充分利用空谱联合提供的语义信息,有利于提取小样本和高维特性的高光谱影像特征。基于双卷积池化结构的3D-CNN网络将没有经过特征处理的3D遥感影像作为输入数据,产生的深度学习分类器模型以端到端的方式训练,不需要做复杂的预处理,此外模型使用了BN和Dropout等正则化策略以避免过拟合现象。结果 实验对比了SVM（support vector machine）、SAE（stack autoencoder）以及目前主流的CNN方法,该模型在Indian Pines和Pavia University数据集上最高分别取得了99.65%和99.82%的总体分类精度,有效提高了高光谱遥感影像地物分类精度。结论 讨论了双卷积池化结构的数目、正则化策略、高光谱首层卷积的光谱采样步长、卷积核大小、相邻像素块大小和学习率等6个因素对实验结果的影响,本文提出的双卷积池化结构可以根据数据集特点进行组合复用,与其他深度学习模型相比,需要更少的参数,计算效率更高。     

一种基于谱聚类的半监督聚类方法

半监督聚类利用少部分标签的数据辅助大量未标签的数据进行非监督的学习,从而提高聚类的性能。提出一种基于谱聚类的半监督聚类算法,其利用标签数据的信息,调整点与点之间的距离所形成的距离矩阵,而后基于被调整的距离矩阵进行谱聚类。实验表明,该算法较之于已提出的半监督聚类算法,获得了更好的聚类性能。     

Self-Learning of Multivariate Time Series Using Perceptually Important Points

In machine learning, positive-unlabelled (PU) learning is a special case within semi-supervised learning. In positive-unlabelled learning, the training set contains some positive examples and a set of unlabelled examples from both the positive and negative classes. Positive-unlabelled learning has gained attention in many domains, especially in time-series data, in which the obtainment of labelled data is challenging. Examples which originate from the negative class are especially difficult to acquire. Self-learning is a semi-supervised method capable of PU learning in time-series data. In the self-learning approach, observations are individually added from the unlabelled data into the positive class until a stopping criterion is reached. The model is retrained after each addition with the existent labels. The main problem in self-learning is to know when to stop the learning. There are multiple, different stopping criteria in the literature, but they tend to be inaccurate or challenging to apply. This publication proposes a novel stopping criterion, which is called Peak evaluation using perceptually important points, to address this problem for time-series data. Peak evaluation using perceptually important points is exceptional, as it does not have tunable hyperparameters, which makes it easily applicable to an unsupervised setting. Simultaneously, it is flexible as it does not make any assumptions on the balance of the dataset between the positive and the negative class.      

基于主动深度学习的高光谱影像分类

针对当前高光谱遥感影像分类人工标注样本费时费力,大量未标注样本未得到有效利用以及主要利用光谱信息而忽视空间信息等问题,提出了一种空-谱信息与主动深度学习相结合的高光谱影像分类方法。首先利用主成分分析对原始影像进行降维,在此基础上提取像素的一正方形小邻域作为该像素的空间信息并结合其原始光谱信息得到空谱特征。然后,通过稀疏自编码器得到原始数据的稀疏特征表达,并通过逐层无监督学习稀疏自编码器构建深度神经网络,输出原始数据的深度特征,将其连接到softmax分类器,利用少量标记样本以监督学习的方式完成模型的精调。最后,利用主动学习算法选择最不确定性样本对其进行标注,并加入至训练样本以提高分类器的分类效果。分别对PaviaU影像和PaviaC影像进行分类实验的结果表明,该方法在少量标记样本情况下,相对于传统方法能有效地提高分类精度。