Efficient single image super-resolution via graph-constrained least squares regression

We explore in this paper an efficient algorithmic solution to single image super-resolution (SR). We propose the gCLSR, namely graph-Constrained Least Squares Regression, to super-resolve a high-resolution (HR) image from a single low-resolution (LR) observation. The basic idea of gCLSR is to learn a projection matrix mapping the LR image patch to the HR image patch space while preserving the intrinsic geometric structure of the original HR image patch manifold. Even if gCLSR resembles other manifold learning-based SR methods in preserving the local geometric structure of HR and LR image patch manifolds, the innovation of gCLSR lies in that it preserves the intrinsic geometric structure of the original HR image patch manifold rather than the LR image patch manifold, which may be contaminated by image degeneration (e.g., blurring, down-sampling and noise). Upon acquiring the projection matrix, the target HR image can be simply super-resolved from a single LR image without the need of HR-LR training pairs, which favors resource-limited applications. Experiments on images from the public database show that gCLSR method can achieve competitive quality as state-of-the-art methods, while gCLSR is much more efficient in computation than some state-of-the-art methods.     

结合支持向量回归和图像自相似的单幅图像超分辨率算法

目的 基于学习的单幅图像超分辨率算法是借助实例训练库由一幅低分辨率图像产生高分辨率图像。提出一种基于图像块自相似性和对非线性映射拟合较好的支持向量回归模型的单幅超分辨率方法,该方法不需使用外部图像训练库。方法 首先根据输入的低分辨率图像建立图像金字塔及包含低/高分辨率图像块对的集合;然后在低/高分辨率图像块对的集合中寻找与输入低分辨率图像块的相似块,利用支持向量回归模型学习这些低分辨率相似块和其对应的高分辨率图像块的中心像素之间的映射关系,进而得到未知高分辨率图像块的中心像素。结果 为了验证本文设计算法的有效性,选取结构和纹理不同的7幅彩色高分辨率图像,对其进行高斯模糊的2倍下采样后所得的低分辨率图像进行超分辨率重构,与双三次插值、基于稀疏表示及基于支持向量回归这3个超分辨率方法重建的高分辨率图像进行比较,峰值信噪比平均依次提升了2.37 dB、0.70 dB和0.57 dB。结论 实验结果表明,本文设计的算法能够很好地实现图像的超分辨率重构,特别是对纹理结构相似度高的图像具有更好的重构效果。     

基于残差的图像超分辨率重建

提出一种基于图像残差的超分辨率重建算法.以原高分辨率图像与插值放大后图像之间的图像残差与低分辨率图像样本特征作为样本对,对其进行K均值分类,并对每类样本对采用KSVD(K-singular value decomposition)方法进行训练获得高、低分辨率字典对,然后根据测试样本与类中心的欧氏距离选择字典对,以与测试样本相近的多个类别所重建的结果加权获得图像残差,并结合低分辨率图像的插值结果获得高分辨率图像.实验结果表明,提出的方法具有更高的重建质量,且采用训练样本分类和相近类别的重建结果的加权和有利于提高图像重建质量.     

Image super-resolution by estimating the enhancement weight of self example and external missing patches

Image super-resolution (SR) is the process of generating a high-resolution (HR) image using one or more low-resolution (LR) inputs. Many SR methods have been proposed, but generating the small-scale structure of an SR image remains a challenging task. We hence propose a single-image SR algorithm that combines the benefits of both internal and external SR methods. First, we estimate the enhancement weights of each LR-HR image patch pair. Next, we multiply each patch by the estimated enhancement weight to generate an initial SR patch. We then employ a method to recover the missing information from the high-resolution patches and create that missing information to generate a final SR image. We then employ iterative back-projection to further enhance visual quality. The method is compared qualitatively and quantitatively with several state-of-the-art methods, and the experimental results indicate that the proposed framework provides high contrast and better visual quality, particularly for non-smooth texture areas.     

Nuclear norm regularized coding with local position-patch and nonlocal similarity for face hallucination

Learning-based face hallucination methods have received much attention and progress in past few decades. Specially, position-patch based approaches have been proposed to replace the probabilistic graph-based or manifold learning-based ones. As opposed to the existing patch based methods, where the input image patch matrix is converted into vectors before combination coefficients calculation, in this paper, we propose to directly use the image matrix based regression model for combination coefficients computation to preserve the essential structural information of the input patch matrix. For each input low-resolution (LR) patch matrix, its combination coefficients over the training image patch matrices at the same position can be computed. Then the corresponding high-resolution (HR) patch matrix can be obtained with the LR training patches replaced by the corresponding HR ones. The nonlocal self-similarities are finally utilized to further improve the hallucination performance. Various experimental results on standard face databases indicate that our proposed method outperforms some state-of-the-art algorithms in terms of both visual quantity and objective metrics.     

Single image super-resolution based on space structure learning

In this paper, the learning-based single image super-resolution (SR) is regarded as a problem of space structure learning. We propose a new SR method that identifies a space from the low-resolution (LR) image space that best preserves the structure of the high-resolution (HR) image space. The inference between the two structure-consistent spaces proves to be accurate and predicts HR image patches with higher quality. An effective iterative algorithm is also proposed to find the near-optimal solution to the model, which can be easily implemented in parallel computing. Extensive experiments are performed to show the effectiveness of the proposed algorithm.     

Global consistency,local sparsity and pixel correlation: A unified framework for face hallucination

In this paper, a novel two-phase framework is presented to deal with the face hallucination problem. In the first phase, an initial high-resolution (HR) face image is produced in patch-wise. Each input low-resolution (LR) patch is represented as a linear combination of training patches and the corresponding HR patch is estimated by the same combination coefficients. Realizing that training patches similar with the input may provide more appropriate textures in the reconstruction, we regularize the combination coefficients by a weighted _?2-norm${?}_2\mathrm{-}\mathrm{norm}$ minimization term which enlarges the coefficients for relevant patches. The HR face image is then initialized by integrating all the HR patches. In the second phase, three regularization models are introduced to produce the final HR face image. Different from most previous approaches which consider global and local priors separately, the proposed algorithm incorporates the global reconstruction model, the local sparsity model and the pixel correlation model into a unified regularization framework. Initializing the regularization problem with the HR image obtained in the first phase, the final output HR image can be optimized through an iterative procedure. Experimental results show that the proposed algorithm achieves better performances in both reconstruction error and visual quality.     

基于图像超分辨极限学习机的极低分辨率人脸识别

极低分辨率图像本身包含的判别信息少且容易受到噪声的干扰,在现有的人脸识别算法下识别率较低。为了解决这一问题,提出一种基于图像超分辨率(SR)极限学习机(ELM)的人脸识别算法。首先,从样本库学习耦合的高低分辨率图像稀疏表达字典,利用高低分辨率表达系数的流形一致性重建高分辨率图像;其次,在超分辨率重建的高分辨率(HR)图像上构建ELM模型,训练获得前向神经网络的连接权值;最后,通过ELM预测输入极低人脸图像的类别属性。实验结果表明,针对于重建后的极低分辨率人脸图片,与协同表示的分类(CRC)人脸识别算法相比,所提算法将识别率分别提升了2%;同时也大幅度缩短了识别的时间。结果表明所提算法能够有效解决极低分辨率图片判决信息不足的问题,具有较好的识别能力。     

基于非负邻域嵌入和非局部正则化的单帧图像超分辨率重建算法

单帧图像超分辨率重建是指利用一幅低分辨率图像,通过相应的算法来获取一幅高分辨率图像的技术。提出了一种基于 非负邻域嵌入和 非局部正则化 的单帧图像超分辨率重建算法,以弥补传统邻域嵌入算法的不足。在训练阶段,首先对低分辨率图像预放大2倍,以保证在放大倍数较大时,高、低分辨率图像块之间的邻域关系也能得到较好的保持;在重建阶段,使用非负邻域嵌入来有效地解决近邻数的选取问题;最后利用图像块的非局部相似性构造非局部正则项对重建结果进行修正。实验结果表明,相对于传统算法,本方法的重建结果纹理丰富、边缘清晰。     

Single Image Super-Resolution Reconstruction Technique based on A Single Hybrid Dictionary

A new sparse domain approach is proposed in this paper to realize the single image super-resolution (SR) reconstruction based upon one single hybrid dictionary, which is deduced from the mixture of both the high resolution (HR) image patch samples and the low resolution (LR) ones. Moreover, a linear model is proposed to characterize the relationship between the sparse representations of both the HR image patches and the corresponding LR ones over the same hybrid dictionary. It is shown that, the requirement on the identical sparse representation of both HR and LR image patches over the corresponding HR dictionary and the LR dictionary can be relaxed. It is unveiled that, the use of one single hybrid dictionary can not only provide a more flexible framework to keep the similar sparse characteristics between the HR patches and the corresponding degenerated LR patches, but also to accommodate their differences. On this basis, the sparse domain based SR reconstruction problem is reformulated. Moreover, the proposed linear model between the sparse representations of both the HR patch and the corresponding LR patch over the same hybrid dictionary offers us a new method to interpret the image degeneration characteristics in sparse domain. Finally, practical experimental results are presented to test and verify the proposed SR approach.     

数据外补偿的深度网络超分辨率重建

单张图像超分辨率重建受到多对一映射的困扰.对于给定的低分辨率图像块,存在若干高分辨率图像块与之对应.基于学习的方法受此影响,学习到的逆映射规则只能预测这些高分辨率图像块的均值,从而产生视觉上模糊的超分辨率重建结果.为了克服歧义性造成的高频细节损失,本文提出了一种基于深度网络,利用在线检索的数据进行高频信息补偿的图像超分辨率重建算法.该方法构建一个深度网络,通过三个分支预测高分辨率重建结果：一条旁路直接将输入的低分辨率图像输入到网络的最后一层;一条内部高频信息重建路径基于低分辨率图像回归预测高分辨率图像,重建高分辨率图像的主要结构;另一条外部高频信息补偿路径根据内部重建的结果,从在线检索到的相似图像中提取高频细节,对内部重建的重建结果进行细节补偿.在第二条路径中,为了有效提取高频信号并使之适应于内部重建的重建结构,本文在多层特征的测量和约束下,进行高频细节迁移.相比于之前基于云数据库的传统图像超分辨率方法,本文提出的方法是端对端可训练的（end-to-end trainable）,因此通过在大数据上进行学习,方法能同时建模内部重建和外部补偿,并能自动权衡两者利弊而给出最优的重建结果.图像超分辨率重建的实验结果表明,相比于最新的超分辨率算法,本文方法在主客观评价中均取得了更加优越的性能.     

Learning graph-constrained cascade regressors for single image super-resolution

Learning cascade regression has been shown an effective strategy to further enhance the perceptual quality of resulted high-resolution (HR) images. However, previous cascade regression-based SR methods have two obvious weaknesses: (1)edge structures cannot be preserved well when applying texture features to represent low-resolution (LR) images, and (2)the local manifold structures spanned by the LR-HR feature spaces cannot be revealed by the learned local linear mappings. To alleviate the aforementioned problems, a novel example regression-based super-resolution (SR) approach called learning graph-constrained cascade regressors (LGCCR) is presented, which learns a group of multi-round residual regressors in a unique way. Specifically, we improve the edge preservation capability by synthesizing the whole HR image rather than local image patches, which facilitates to extract the edge features to represent LR images. Moreover, we utilize a graph-constrained regression model to build the local linear regressors, where each local linear regressor responds to an anchored atom in the learned over-complete dictionary. Both quantitative and qualitative quality evaluations on seven benchmark databases indicate the superiority of the proposed LGCCR-based SR approach in comparing with other state-of-the-art SR predecessors.

     

Image clustering based on sparse patch alignment framework

Image clustering methods are efficient tools for applications such as content-based image retrieval and image annotation. Recently, graph based manifold learning methods have shown promising performance in extracting features for image clustering. Typical manifold learning methods adopt appropriate neighborhood size to construct the neighborhood graph, which captures local geometry of data distribution. Because the density of data points’ distribution may be different in different regions of the manifold, a fixed neighborhood size may be inappropriate in building the manifold. In this paper, we propose a novel algorithm, named sparse patch alignment framework, for the embedding of data lying in multiple manifolds. Specifically, we assume that for each data point there exists a small neighborhood in which only the points that come from the same manifold lie approximately in a low-dimensional affine subspace. Based on the patch alignment framework, we propose an optimization strategy for constructing local patches, which adopt sparse representation to select a few neighbors of each data point that span a low-dimensional affine subspace passing near that point. After that, the whole alignment strategy is utilized to build the manifold. Experiments are conducted on four real-world datasets, and the results demonstrate the effectiveness of the proposed method.     

Enhancement of Infrared Images Using Super Resolution Techniques Based on Big Data Processing

This paper presents a super-resolution (SR) technique for enhancement of infrared (IR) images. The suggested technique relies on the image acquisition model, which benefits from the sparse representations of low-resolution (LR) and high-resolution (HR) patches of the IR images. It uses bicubic interpolation and minimum mean square error (MMSE) estimation in the prediction of the HR image with a scheme that can be interpreted as a feed-forward neural network. The suggested algorithm to overcome the problem of having only LR images due to hardware limitations is represented with a big data processing model. The performance of the suggested technique is compared with that of the standard regularized image interpolation technique as well as an adaptive block-by-block least-squares (LS) interpolation technique from the peak signal-to-noise ratio (PSNR) perspective. Numerical results reveal the superiority of the proposed SR technique.

     

A regularized pan-sharpening approach based on self-similarity and Gabor prior

In this article, we propose a new regularization-based approach for pan-sharpening based on the concepts of self-similarity and Gabor prior. The given low spatial resolution (LR) and high spectral resolution multi-spectral (MS) image is modelled as degraded and noisy version of the unknown high spatial resolution (HR) version. Since this problem is ill-posed, we use regularization to obtain the final solution. In the proposed method, we first obtain an initial HR approximation of the unknown pan-sharpened image using self-similarity and sparse representation (SR) theory. Using self-similarity, we obtain the HR patches from the given LR observation by searching for matching patches in its coarser resolution, thereby obtaining LR–HR pairs. An SR framework is used to obtain the patch pairs for which no matches are available for the patches in LR observation. The entire set of matched HR patches constitutes initial HR approximation (initial estimate) to the final pan-sharpened image which is used to estimate the degradation matrix as used in our model. A regularization framework is then used to obtain the final solution in which we propose to use a new prior which we refer as Gabor prior that extracts the bandpass details from the registered panchromatic (Pan) image. In addition, we also include Markov random field (MRF) smoothness prior that preserves the smoothness in the final pan-sharpened image. MRF parameter is derived using the initial estimate image. The final cost function consists of data fitting term and two prior terms corresponding to Gabor and MRF. Since the derived cost function is convex, simple gradient-based method is used to obtain the final solution. The efficacy of the proposed method is evaluated by conducting the experiments on degraded as well as on un-degraded datasets of three different satellites, i.e., Ikonos-2, Quickbird, and Worldview-2. The results are compared on the basis of traditional measures as well as recently proposed quality with no reference (QNR) measure, which does not require the reference image.     

Single image super-resolution via multiple linear mapping anchored neighborhood regression

Multimedia Tools and Applications - The goal of learning-based image super-resolution (SR) is to generate a plausible and visually high-resolution (HR) image from a single low-resolution (LR) input...     

基于半耦合稀疏表达的极低分辨率人脸识别

现有基于学习的人脸超分辨率算法假设高低分辨率特征具有流形一致性（耦合字典学习）,然而低分辨率图像的降质过程使得高低分辨率特征产生了“一对多”的映射关系偏差,减少了极低分辨率图像特征的判决信息,降低了超分辨率重建图像的识别率。针对这一问题,引入了半耦合稀疏字典学习模型,松弛高低分辨率流形一致性假设,同时学习稀疏表达字典和稀疏表达系数之间的映射函数,提升高低分辨率判决特征的一致性,在此基础上,引入协同分类模型,实现半耦合特征的高效分类。实验表明：相比于传统稀疏表达分类算法,算法不仅提高了识别率,并且还大幅度降低了时间开销,验证了半耦合稀疏学习字典在人脸识别中的有效性。     

Video super-resolution network using detail component extraction and optical flow enhancement algorithm

The video super-resolution (SR) task refers to the use of corresponding low-resolution (LR) frames and multiple neighboring frames to generate high-resolution (HR) frames. Existing deep learning-based approaches usually utilize LR optical flow for video SR tasks. However, the accuracy of LR optical flow is not enough to recover the fine detail part. In this paper, we propose a video SR network that uses optical flow SR and optical flow enhancement algorithms to provide accurate temporal dependency. And extract the detail component of LR adjacent frames as supplementary information for accurate feature extraction. Firstly, the network infers HR optical flow from LR optical flow, and uses the optical flow enhancement algorithm to enhance HR optical flow. Then the processed HR optical flows are used as the input of the motion compensation network. Secondly, we extract detail component to reduce the error caused by motion compensation based on optical flow. Finally, the SR results are generated through the SR network. We perform comprehensive comparative experiments on two datasets: Vid4 and DAVIS. The results show that, compared with other state-of-the-art methods, the proposed video SR method achieves the better performance.

     

New edge-adaptive image interpolation using anisotropic Gaussian filters

Edge-adaptive interpolation is an effective technique to obtain a sharp high-resolution (HR) image from a low-resolution (LR) image. In this paper, we propose a new image interpolation method using edge-adaptive anisotropic Gaussian filters (AGFs). In the proposed method, the edge information of the LR image, which is the edge orientation and its energy, is first estimated using the modified Leung–Malik (LM) filter bank. Then, the edge information of the LR image is converted into that of the HR image by using a mapping function. Based on the HR edge information, edge-adaptive interpolation is performed using the AGF with variable shape and direction. Simulation results show that the proposed method objectively and subjectively outperforms conventional edge-based methods.     

Multi-resolution feature fusion for face recognition

For face recognition, image features are first extracted and then matched to those features in a gallery set. The amount of information and the effectiveness of the features used will determine the recognition performance. In this paper, we propose a novel face recognition approach using information about face images at higher and lower resolutions so as to enhance the information content of the features that are extracted and combined at different resolutions. As the features from different resolutions should closely correlate with each other, we employ the cascaded generalized canonical correlation analysis (GCCA) to fuse the information to form a single feature vector for face recognition. To improve the performance and efficiency, we also employ “Gabor-feature hallucination”, which predicts the high-resolution (HR) Gabor features from the Gabor features of a face image directly by local linear regression. We also extend the algorithm to low-resolution (LR) face recognition, in which the medium-resolution (MR) and HR Gabor features of a LR input image are estimated directly. The LR Gabor features and the predicted MR and HR Gabor features are then fused using GCCA for LR face recognition. Our algorithm can avoid having to perform the interpolation/super-resolution of face images and having to extract HR Gabor features. Experimental results show that the proposed methods have a superior recognition rate and are more efficient than traditional methods.