高光谱图像超分辨率重建技术研究进展

不同于传统图像（如灰度图像、RGB图像等）专注于保存目标场景的空间信息，高光谱图像蕴含丰富的空—谱信息，不仅可以保存目标的空间信息，还可以保存具有高可辨性的光谱信息。因此高光谱图像广泛应用于多种计算机视觉和遥感图像任务中，如目标检测、场景分类和目标追踪等。然而，在高光谱图像获取以及重建过程中仍然存在许多问题与瓶颈。如传统高光谱成像仪器在成像过程中通常会引入噪声，且获得的图像往往具有较低的空间分辨率，极大地影响了高光谱图像的质量，对后续数据分析任务造成了极大的困难。近年来，高光谱图像超分辨率重建技术研究得到了极大的发展，现有超分辨率重建方法可以大致分为两类，一类为空间超分辨率重建方法，可以通过直接提升高光谱图像的空间分辨率来获得高质量高光谱图像；另一类为光谱超分辨率重建方法，可以通过提升高空间分辨率图像的光谱分辨率来生成高质量高光谱图像。本文从高光谱图像超分辨率重建领域的新设计、新方法和应用场景出发，通过综合国内外前沿文献来梳理该领域的主要发展，重点论述高光谱图像超分辨率重建领域的发展现状、前沿动态、热点问题及趋势。

A survey of hyperspectral image super-resolution method

Computer vision-oriented hyperspectral images(HSIs) are featured to enrich spatial and spectral information compared to gray and RGB images. It has been developing in such domains like target detection, scene classification, and tracking. However, the HSI imaging technique is challenged for its distortion problems(e. g., low spatial resolution, noise). HSI-related super-resolution(SR) methods are proposed to reconstruct high-quality HSIs in terms of a high spectral resolution and spatial resolution(HR). Current HSI SR methods can be segmented into two categories:spatial SR and spectral SR. The spatial SR method is oriented to reconstruct the target HR HSI via improving the spatial resolution of low-resolution(LR) HSI. It can be subdivided into single image SR and fusion based SR methods further. Single image based SR method can be used to reconstruct the target HSI via directly improving the spatial resolution of LR HSI. However, due to much more spatial information is sacrificed, single image based SR method is challenged to reconstruct effective HSIs.Therefore, to fuse these high quality spatial information into the LR HSIs, extra high spatial homogeneous information is introduced(e. g., multispectral image(MSI), RGB). In this way, the spatial resolution of LR HSI can be improved greatly(e. g., 8 times, 16 times, and 32 times SR). The other HSI SR method is focused on the spectral super-resolution method, which can improve the spectral resolution of high spatial resolution images(e. g., MSI, RGB) and generate the target hyperspectral image. The literature review is focused on the growth of HSI SR methods in relevance to three aspects single image-based, fusion-based, and spectral super-resolution contexts. Each of these three category for HSI SR methods can be further subdivided into two aspects of traditional optimization framework and deep learning based methods. For single image HSI SR, due to the SR problem being an illness inverse problem, traditional optimization framework based single image HSI SR methods is used to develop effective image priors to restrain the SR process. Image priors like low-rank, sparse representation, and non-local features are commonly used in single based HSI SR method, but it still challenged for such problems of manpower and restrictions. For the traditional fusion based method, the core element is to balance the spatial-spectral correlation between HR MSI and LR HSI. It is feasible to split these two images into key components and then re-combine the effective parts of each image. Therefore, multiple schemes are introduced(e. g., nonnegative matrix factorization, coupled tensor factorization) to leverage the key information from HR MSI and LR HSI. In addition, to increase the effectiveness of these decomposition methods, some constraints are required to be introduced (e. g., sparse, low-rank). For the traditional spectral resolution method, it is essential to learn how to reconstruct the spectral characteristics from RGB/MSI images. When paired RGB and HSI exist, a promising way is feasible to construct a dictionary(e. g., sparse dictionary learning) that the mapping relation can be recorded between RGB/MSI images and HSIs. The dictionary learning based spectral super-resolution methods have been developing as well, but it is often challenged to realize more generalization ability for applications. In recent years, deep learning based methods have facilitated much more computer vision tasks, and is beneficial for exploiting the inherent spatial-spectral relations of HSIs. For the single image HSI SR methods, a deep convolution neural network(DCNN) is utilized to learn the mapping process from LR HSI to HR HSI. The DCNN is capable to learn deep image prior from plenty of training samples, which has better representation ability than the heuristic handcrafted image priors(e. g., low-rank, sparse) to some extent. However, the performance of this kind of method is often restricted by the amount and variety of training samples. Therefore, it is necessary to reconstruct HR HSI from LR HSI in an unsupervised manner, but the robustness of the unsupervised method is still a challenging problem to be resolved. For the deep learning based HSI fusion methods, to extract spatial-spectral information from MSIs and HSIs, an effective design of DCNN frameworks(e. g., multi-branch, multi-scale, 3D-CNN) are concerned about. However, such problems(e. g., noise, unknown degeneration, unregistered) are challenged for effective generalization ability of DCNN-based HSI fusion method. To resolve these problems, unsupervised and alternative learning methods are introduced to improve fusion-based generalization ability. The DCNN-unfolding is proposed and the optimal interpretability is demonstrated, and its registration strategy is introduced to improve the robustness further. For the spectral super resolution method, DCNN is demonstrated to model the mapping scheme between RGB/MSI and HR HSI. However, there still some barriers are required to be resolved for DCNN-based spectral super resolution method. For example, most existing spectral super resolution methods can generate HSI only according to its fixed spectral interval or spectrum range. A spectral super resolution framework is required to be linked with generalization ability well in the future. In this study, we will summarize the developing of the HSI SR scope from the perspective of new designs, new methods, and new application scenes.