High resolution image formation from low resolution frames using Delaunay triangulation

An algorithm based on spatial tessellation and approximation of each triangle patch in the Delaunay (1934) triangulation (with smoothness constraints) by a bivariate polynomial is advanced to construct a high resolution (HR) high quality image from a set of low resolution (LR) frames. The high resolution algorithm is accompanied by a site-insertion algorithm for update of the initial HR image with the availability of more LR frames till the desired image quality is attained. This algorithm, followed by post filtering, is suitable for real-time image sequence processing because of the fast expected (average) time construction of Delaunay triangulation and the local update feature.     

Super-resolution image reconstruction: a technical overview

A new approach toward increasing spatial resolution is required to overcome the limitations of the sensors and optics manufacturing technology. One promising approach is to use signal processing techniques to obtain an high-resolution (HR) image (or sequence) from observed multiple low-resolution (LR) images. Such a resolution enhancement approach has been one of the most active research areas, and it is called super resolution (SR) (or HR) image reconstruction or simply resolution enhancement. In this article, we use the term "SR image reconstruction" to refer to a signal processing approach toward resolution enhancement because the term "super" in "super resolution" represents very well the characteristics of the technique overcoming the inherent resolution limitation of LR imaging systems. The major advantage of the signal processing approach is that it may cost less and the existing LR imaging systems can be still utilized. The SR image reconstruction is proved to be useful in many practical cases where multiple frames of the same scene can be obtained, including medical imaging, satellite imaging, and video applications. The goal of this article is to introduce the concept of SR algorithms to readers who are unfamiliar with this area and to provide a review for experts. To this purpose, we present the technical review of various existing SR methodologies which are often employed. Before presenting the review of existing SR algorithms, we first model the LR image acquisition process.     

Hypergraph-regularized sparse representation for single color image super resolution

Sparsity-based single image super resolution method generates the High-Resolution (HR) output via a corresponding dictionary from the Low-Resolution (LR) input. However, most of these existing methods ignore the complementary information from color channels, which causes the loss of a valid prior and the limitation of HR image quality improvement. In this paper, hypergraph regularization is first incorporated with Joint Color Dictionary Training (JCDT) model and HR image reconstruction (HRIR) model. A novel Hypergraph-regularized Sparse coding-based Super Resolution (HG-ScSR) is proposed. This regularization can not only focus on the illuminance information, but also exploit the self-channel and cross-channel information of three color RGB channels from high-resolution image patches. Especially, the complex relationship is explored among every color image patch pixel and the consistency of the similar pixels is enforced. Both simulated and real data experiments verify the higher performance of the proposed HG-ScSR.     

An image super-resolution algorithm for different error levels per frame.

In this paper, we propose an image super-resolution (resolution enhancement) algorithm that takes into account inaccurate estimates of the registration parameters and the point spread function. These inaccurate estimates, along with the additive Gaussian noise in the low-resolution (LR) image sequence, result in different noise level for each frame. In the proposed algorithm, the LR frames are adaptively weighted according to their reliability and the regularization parameter is simultaneously estimated. A translational motion model is assumed. The convergence property of the proposed algorithm is analyzed in detail. Our experimental results using both real and synthetic data show the effectiveness of the proposed algorithm.     

一种从低分辨率图像序列获取高分辨率图像的算法

在某些应用领域,常常需要得到目标的高分辨率图像,考虑到在这些应用场合中,往往可以获得对同一景物或目标的多帧图像,本文提出了一种基于亚像素级图像配准与类似于中值滤波插值的从多幅低分辨率(LR)图像中获取一幅高分辨率(HR)图像的算法。算法首先采用梯度方法计算出LR图像之间的位移量,经过图像配准后,每个HR像素点被赋予其作用域内所有LR像点值的中值。仿真结果表明,该算法简单有效,既能提高图像分辨率,又能较好地去除非线性噪声。     

Mathematical analysis of super-resolution methodology

The attainment of super resolution (SR) from a sequence of degraded undersampled images could be viewed as reconstruction of the high-resolution (HR) image from a finite set of its projections on a sampling lattice. This can then be formulated as an optimization problem whose solution is obtained by minimizing a cost function. The approaches adopted and their analysis to solve the formulated optimization problem are crucial, The image acquisition scheme is important in the modeling of the degradation process. The need for model accuracy is undeniable in the attainment of SR along with the design of the algorithm whose robust implementation will produce the desired quality in the presence of model parameter uncertainty. To keep the presentation focused and of reasonable size, data acquisition with multisensors instead of, say a video camera is considered.     

图约束字典和加权稀疏表示人脸超分辨率算法

针对低分辨率、低质量人脸图像重建问题,提出了一种新的基于稀疏表示的人脸超分辨率算法。在训练阶段,人脸的位置特征被用于保持人脸块的全局信息,人脸块间的几何结构被用于保持高低分辨率超完备冗余字典的流形结构,从而提高字典的表达能力;在重建阶段,K近邻加权稀疏表示被用于消除稀疏编码噪声,以提高高分辨率人脸图像重建系数的精度。实验结果表明,提出的方法取得了较好的主客观质量。     

一种新的基于MAP的纹理自适应超分辨率图像复原算法

 本文论证了超分辨率图像复原计算中的两个性质,并基于此在MAP(Maximum A Posteriori)框架下提出了一种新的纹理自适应算法.算法首先根据低分辨率图像和高分辨率图像近似计算的可类比性质计算初始图像,使初始图像的质量更高,并根据超分辨率复原图像阶跃边缘的陡坡性质,将三边滤波正则化应用于迭代运算中,更好地保护了图像的陡坡和屋顶边缘.算法可根据图像的纹理自动计算初始图像融合参数以及正则化函数中的梯度阈值等参数,解决了以往超分辨率图像复原算法参数调整复杂的问题.实验结果表明,本文算法在没有人工参与的情况下,重建图像的客观评价和主观质量均有明显提高.     

A fast image super-resolution algorithm using an adaptive Wiener filter.

A computationally simple super-resolution algorithm using a type of adaptive Wiener filter is proposed. The algorithm produces an improved resolution image from a sequence of low-resolution (LR) video frames with overlapping field of view. The algorithm uses subpixel registration to position each LR pixel value on a common spatial grid that is referenced to the average position of the input frames. The positions of the LR pixels are not quantized to a finite grid as with some previous techniques. The output high-resolution (HR) pixels are obtained using a weighted sum of LR pixels in a local moving window. Using a statistical model, the weights for each HR pixel are designed to minimize the mean squared error and they depend on the relative positions of the surrounding LR pixels. Thus, these weights adapt spatially and temporally to changing distributions of LR pixels due to varying motion. Both a global and spatially varying statistical model are considered here. Since the weights adapt with distribution of LR pixels, it is quite robust and will not become unstable when an unfavorable distribution of LR pixels is observed. For translational motion, the algorithm has a low computational complexity and may be readily suitable for real-time and/or near real-time processing applications. With other motion models, the computational complexity goes up significantly. However, regardless of the motion model, the algorithm lends itself to parallel implementation. The efficacy of the proposed algorithm is demonstrated here in a number of experimental results using simulated and real video sequences. A computational analysis is also presented.     

带总观测误差约束的模糊图像恢复

模糊图像恢复是数字图像处理领域的研究热点之一,总变差(Total Variation, TV)规整化可以很好的保持图像的细节,然而,传统的TV图像恢复模型需要考虑最优的正则化参数,由此,提出了一族包含不同规整化因子,带总观测误差约束的模糊图像恢复模型,并分为去模糊和去噪两步求解此模型。在去模糊过程中,利用共轭梯度法求出一个满足总观测误差约束的初始恢复图像;在去噪过程中,首先,以去模糊的结果作为初始估计;其次,针对 范数最小化问题,利用优化—最小化(Majoriziation-Minimization, MM)算法的思想,将原问题转化为一系列容易求解的优化子问题;最后,极小化优化子问题,得到最终的恢复图像。实验结果表明,该算法对模糊图像的恢复效果是显著地。      

Video super resolution based on non-local regularization and reliable motion estimation

Video super-resolution (SR) is a process for reconstructing high-resolution (HR) images by utilizing complementary information among multiple low-resolution (LR) images. Accurate estimation of the motion among the LR images significantly affects the quality of the reconstructed HR image. In this paper, we analyze the possible reasons for the inaccuracy of motion estimation and then propose a multi-lateral filter to regularize the process of motion estimation. This filter can adaptively correct motion estimation according to the estimation reliability, image intensity discontinuity, and motion dissimilarity. Furthermore, we introduce a non-local prior to solve the ill-posed problem of HR image reconstruction. This prior can fully utilize the self-similarities existing in natural images to regularize the HR image reconstruction. Finally, we employ a Bayesian formulation to incorporate the two regularizations into one Maximum a Posteriori (MAP) estimation model, where the HR image and the motion estimation can be refined progressively in an alternative and iterative manner. In addition, an algorithm that estimates the blur kernel by analyzing edges in an image is also presented in this paper. Experimental results demonstrate that the proposed approaches are highly effective and compare favorably to state-of-the-art SR algorithms.     

Blind single-image super resolution based on compressive sensing

Blind super resolution is an interesting area in image processing that can restore high resolution (HR) image without requiring prior information of the volatile point spread function (PSF). In this paper, a novel framework is proposed for blind single-image super resolution (SISR) problem based on compressive sensing (CS) framework that is one of the first works that considers general PSFs. The fundamental idea in the proposed approach is to use sparsity on a known sparse transform domain as a powerful regularizer in both the image and blur domains. Therefore, a new cost function with respect to the unknown HR image patch and PSF kernel is presented and minimization is performed based on two subproblems that are modeled similar to that of CS. Simulation results demonstrate the effectiveness of the proposed algorithm that is competitive with methods that use multiple LR images to achieve a single HR image.     

Pan-sharpening via multi-scale and multiple deep neural networks

Interpreting remote sensing images by combining manual visual interpretation and computer automatic classification and recognition is an important application of human–computer interaction (HCI) in the field of remote sensing. Remote sensing images with high spatial resolution and high spectral resolution is an important basis for automatic classification and recognition. However, such images are often difficult to obtain directly. In order to solve the problem, a novel pan-sharpening method via multi-scale and multiple deep neural networks is presented. First, the non-subsampled contourlet transform (NSCT) is employed to decompose the high resolution (HR)/low resolution (LR) panchromatic (PAN) images into the high frequency (HF)/low frequency (LF) images, respectively. For pan-sharpening, the training sets are only sampled from the HF images. Then, the DNN is utilized to learn the feature of the HF images in different directions of HR/LR PAN images, which is trained by the image patch pair sampled from HF images of HR/LR PAN images. Moreover, in the fusion stage, NSCT is also employed to decompose the principal component of initially amplified LR multispectral (MS) image obtained by the transformation of adaptive PCA (A-PCA). The HF image patches of LR MS, as the input data of the trained DNN, go through forward propagation to obtain the output HR MS image. Finally, the output HF sub-band images and the original LF sub-band images of LR MS image fuse into a new sub-band set. The inverse transformations of NSCT and A-PCA , residual compensation are conducted to obtain the pan-sharpened HR MS. The experimental results show that our method is better than other well-known pan-sharpening methods.     

A nonlinear least square technique for simultaneous image registration and super-resolution.

This paper proposes a new algorithm to integrate image registration into image super-resolution (SR). Image SR is a process to reconstruct a high-resolution (HR) image by fusing multiple low-resolution (LR) images. A critical step in image SR is accurate registration of the LR images or, in other words, effective estimation of motion parameters. Conventional SR algorithms assume either the estimated motion parameters by existing registration methods to be error-free or the motion parameters are known a priori. This assumption, however, is impractical in many applications, as most existing registration algorithms still experience various degrees of errors, and the motion parameters among the LR images are generally unknown a priori. In view of this, this paper presents a new framework that performs simultaneous image registration and HR image reconstruction. As opposed to other current methods that treat image registration and HR reconstruction as disjoint processes, the new framework enables image registration and HR reconstruction to be estimated simultaneously and improved progressively. Further, unlike most algorithms that focus on the translational motion model, the proposed method adopts a more generic motion model that includes both translation as well as rotation. An iterative scheme is developed to solve the arising nonlinear least squares problem. Experimental results show that the proposed method is effective in performing image registration and SR for simulated as well as real-life images.     

Multiframe super-resolution based on half-quadratic prior with artifacts suppress

The multiframe super-resolution (SR) technique aims to obtain a high-resolution (HR) image by using a set of observed low-resolution (LR) images. In the reconstruction process, artifacts may be possibly produced due to the noise, especially in presence of stronger noise. In order to suppress artifacts while preserving discontinuities of images, in this paper a multiframe SR method is proposed by involving the reconstruction properties of the half-quadratic prior model together with the quadratic prior model using a convex combination. Moreover, by analyzing local features of the underlined HR image, these two prior models are combined by using an automatically calculated weight function, making both smooth and discontinuous pixels handled properly. A variational Bayesian inference (VBF) based algorithm is designed to efficiently and effectively seek the solution of the proposed method. With the VBF framework, motion parameters and hyper-parameters are all determined automatically, leading to an unsupervised SR method. The efficiency of the hybrid prior model is demonstrated theoretically and practically, which shows that our SR method can obtain better results from LR images even with stronger noise. Extensive experiments on several visual data have demonstrated the efficacy and superior performance of the proposed algorithm, which can not only preserve image details but also suppress artifacts.     

基于变分的多尺度遥感图像融合算法

全色图像与多光谱图像融合时,忽略了上采样的重要性和通道间细节的差异性.针对前者,利用不同尺度下自相似块,估计出低分辨率图像丢失信息,从而修改了图像上采样的策略,并以此构造目标函数的保真项;针对后者,利用全色图像和光谱图像梯度域结构相似性,提出局部加权动态稀疏约束,构造目标函数的正则项.本文基于变分法理论,构造了新的目标函数,并提出了多尺度迭代融合框架,通过多次迭代逐步提高融合图像的分辨率,每一层的融合结果更加准确,从而提高最终的融合精度.本文算法与Brovey等成分替代算法、P+XS等变分算法、MTF_GLP等多分辨分析算法进行比较.实验结果表明,本算法的融合结果具有良好的视觉效果,且在客观评价指标上比所有对比算法的最优值平均值均有提高.     

Adaptive Least Squares Interpolation of Infrared Images

This paper presents an adaptive block-by-block least squares (LS) algorithm for the interpolation of infrared (IR) images. The suggested algorithm is based on the segmentation of the low resolution (LR) image into overlapping blocks and the interpolation of each block, separately. The purpose of the overlapping is to avoid edge effects between blocks. An iterative implementation of the proposed algorithm, which considers the image acquisition model, is used for the minimization of the estimation error in each block. A weight matrix of moderate dimensions is estimated in a small number of iterations to interpolate each block. This proposed algorithm avoids the large computational complexity resulting from the matrices of large dimensions required to interpolate the image as a whole. The performance of the proposed algorithm is compared with the standard as well as the warped distance optimal interpolation of maximal order with minimal support (O-MOMS) algorithm from the peak signal-to-noise ratio (PSNR) point of view. Numerical results reveal the superiority of the proposed LS algorithm to the cubic O-MOMS algorithm.     

自适应聚类的全耦合稀疏学习算法

在图像处理领域,基于稀疏表示理论的图像超分辨力算法、高低分辨力字典与稀疏编码之间的映射关系是其中的2个关键环节。由于丰富多样的图像类型,单一字典并不能很好地表示图像。而在稀疏编码之间的映射关系上,严格相等的约束关系也限制了图像重建的效果。针对上述两个方面,采用包容性更强的多个字典与约束条件更为宽松的全耦合稀疏关系进行图像的超分辨力重建。在图像非局部自相似性的基础上,进行多次自适应聚类;挑选出最优的聚类,通过全耦合稀疏学习的图像超分辨力算法,得到多个字典;最后,对输入的低分辨力图像进行分类重建,得到高分辨力图片。实验结果表明,在图像Leaves,Barbara,Room上,本文的聚类算法比原全耦合稀疏学习算法在峰值信噪比(PSNR)上分别提升了0.51 dB,0.21 dB,0.15 dB。     

基于循环生成对抗网络的超分辨率重建算法研究

为了提高图像超分辨率重建的效果,该文将注意力机制引入多级残差网络(Multi-level Residual Attention Network,MRAN)作为CycleGAN的重建网络,提出了基于循环生成对抗网络(CycleGAN)的超分辨率重建模型MRA-GAN.MRA-GAN模型中重建网络负责将低分辨率(LR)图像...