Learning adaptive interpolation kernels for fast single-image super resolution

This paper presents a fast single-image super-resolution approach that involves learning multiple adaptive interpolation kernels. Based on the assumptions that each high-resolution image patch can be sparsely represented by several simple image structures and that each structure can be assigned a suitable interpolation kernel, our approach consists of the following steps. First, we cluster the training image patches into several classes and train each class-specific interpolation kernel. Then, for each input low-resolution image patch, we select few suitable kernels of it to make up the final interpolation kernel. Since the proposed approach is mainly based on simple linear algebra computations, its efficiency can be guaranteed. And experimental comparisons with state-of-the-art super-resolution reconstruction algorithms on simulated and real-life examples can validate the performance of our proposed approach.     

Short kernel fifth-order interpolation

An explicit analytical formula for a short kernel fifth-order polynomial interpolator is obtained. It is also possible to obtain the explicit forms of even higher order interpolation kernels with the method of calculation used, but it is seen that these local kernels become “remainder-dominated” as the order increases. The frequency domain properties and the accuracies of the obtained kernel and the known convolution kernels are compared. Frequency domain comparison with the cubic B-spline interpolators is also given. Some cases of proper use of the calculated kernels have been pointed out     

General-form 3-3-3 interpolation kernel and its simplified frequency-response derivation

An interpolation kernel is required in a wide variety of signal processing applications such as image interpolation and timing adjustment in digital communications. This article presents a general-form interpolation kernel called 3-3-3 interpolation kernel and derives its frequency response in a closed-form by using a simple derivation method. This closed-form formula is preliminary to designing various 3-3-3 interpolation kernels subject to a set of design constraints. The 3-3-3 interpolation kernel is formed through utilising the third-degree piecewise polynomials, and it is an even-symmetric function. Thus, it will suffice to consider only its right-hand side when deriving its frequency response. Since the right-hand side of the interpolation kernel contains three piecewise polynomials of the third degree, i.e. the degrees of the three piecewise polynomials are (3,3,3), we call it the 3-3-3 interpolation kernel. Once the general-form frequency-response formula is derived, we can systematically formulate the design of various 3-3-3 interpolation kernels subject to a set of design constraints, which are targeted for different interpolation applications. Therefore, the closed-form frequency-response expression is preliminary to the optimal design of various 3-3-3 interpolation kernels. We will use an example to show the optimal design of a 3-3-3 interpolation kernel based on the closed-form frequency-response expression.     

Image interpolation by two-dimensional parametric cubic convolution.

Cubic convolution is a popular method for image interpolation. Traditionally, the piecewise-cubic kernel has been derived in one dimension with one parameter and applied to two-dimensional (2-D) images in a separable fashion. However, images typically are statistically nonseparable, which motivates this investigation of nonseparable cubic convolution. This paper derives two new nonseparable, 2-D cubic-convolution kernels. The first kernel, with three parameters (designated 2D-3PCC), is the most general 2-D, piecewise-cubic interpolator defined on [-2, 2] x [-2, 2] with constraints for biaxial symmetry, diagonal (or 90 degrees rotational) symmetry, continuity, and smoothness. The second kernel, with five parameters (designated 2D-5PCC), relaxes the constraint of diagonal symmetry, based on the observation that many images have rotationally asymmetric statistical properties. This paper also develops a closed-form solution for determining the optimal parameter values for parametric cubic-convolution kernels with respect to ensembles of scenes characterized by autocorrelation (or power spectrum). This solution establishes a practical foundation for adaptive interpolation based on local autocorrelation estimates. Quantitative fidelity analyses and visual experiments indicate that these new methods can outperform several popular interpolation methods. An analysis of the error budgets for reconstruction error associated with blurring and aliasing illustrates that the methods improve interpolation fidelity for images with aliased components. For images with little or no aliasing, the methods yield results similar to other popular methods. Both 2D-3PCC and 2D-5PCC are low-order polynomials with small spatial support and so are easy to implement and efficient to apply.     

Spatial domain filtering for fast modification of the tradeoff between image sharpness and pixel noise in computed tomography

In computed tomography (CT), selection of a convolution kernel determines the tradeoff between image sharpness and pixel noise. For certain clinical applications it is desirable to have two or more sets of images with different settings. So far, this typically requires reconstruction of several sets of images. We present an alternative approach using default reconstruction of sharp images and online filtering in the spatial domain allowing modification of the sharpness-noise tradeoff in real time. A suitable smoothing filter function in the frequency domain is the ratio of smooth and original (sharp) kernel. Efficient implementation can be achieved by a Fourier transform of this ratio to the spatial domain. Separating the two-dimensional spatial filtering into two subsequent one-dimensional filtering stages in the x and y directions using a Gaussian approximation for the convolution kernel further reduces computational complexity. Due to efficient implementation, interactive modification of the filter settings becomes possible, which can completely replace the variety of different reconstruction kernels.     

Polyphase antialiasing in resampling of images.

Changing resolution of images is a common operation. It is also common to use simple, i.e., small, interpolation kernels satisfying some "smoothness" qualities that are determined in the spatial domain. Typical applications use linear interpolation or piecewise cubic interpolation. These are popular since the interpolation kernels are small and the results are acceptable. However, since the interpolation kernel, i.e., impulse response, has a finite and small length, the frequency domain characteristics are not good. Therefore, when we enlarge the image by a rational factor of (L/M), two effects usually appear and cause a noticeable degradation in the quality of the image. The first is jagged edges and the second is low-frequency modulation of high-frequency components, such as sampling noise. Both effects result from aliasing. Enlarging an image by a factor of (L/M) is represented by first interpolating the image on a grid L times finer than the original sampling grid, and then resampling it every M grid points. While the usual treatment of the aliasing created by the resampling operation is aimed toward improving the interpolation filter in the frequency domain, this paper suggests reducing the aliasing effects using a polyphase representation of the interpolation process and treating the polyphase filters separately. The suggested procedure is simple. A considerable reduction in the aliasing effects is obtained for a small interpolation kernel size. We discuss separable interpolation and so the analysis is conducted for the one-dimensional case.     

基于非均匀采样重构的图像插值算法

在大规模集成电路实现图像缩放时,通常使用的算法是插值。基于信号的非均匀采样重构算法,提出了一种新的图像插值算法[1]。该插值算法所使用的基函数是基于余弦和函数。本算法同其他算法相比,在插值基函数长度相同的情况下,本算法的性能最好。相较其他算法,本算法保持了图像中更多的高频分量,同时减少了插值过程中的图像混淆效应。同时并没有增加插值所需的运算资源。     

Restoration and reconstruction of AVHRR images

Describes the design of small convolution kernels for the restoration and reconstruction of Advanced Very High Resolution Radiometer (AVHRR) images. The kernels are small enough to be implemented efficiently by convolution, yet effectively correct degradations and increase apparent resolution. The kernel derivation is based on a comprehensive, end-to-end system model that accounts for scene statistics, image acquisition blur, sampling effects, sensor noise, and postfilter reconstruction. The design maximizes image fidelity subject to explicit constraints on the spatial support and resolution of the kernel. The kernels can be designed with finer resolution than the image to perform partial reconstruction for geometric correction and other remapping operations. Experiments demonstrate that small kernels yield fidelity comparable to optimal unconstrained filters with less computation     

Small convolution kernels for high-fidelity image restoration

An algorithm is developed for computing the mean-square-optimal values for small, image-restoration kernels. The algorithm is based on a comprehensive, end-to-end imaging system model that accounts for the important components of the imaging process: the statistics of the scene, the point-spread function of the image-gathering device, sampling effects, noise, and display reconstruction. Subject to constraints on the spatial support of the kernel, the algorithm generates the kernel values that restore the image with maximum fidelity, that is, the kernel minimizes the expected mean-square restoration error. The algorithm is consistent with the derivation of the spatially unconstrained Wiener filter, but leads to a small, spatially constrained kernel that, unlike the unconstrained filter, can be efficiently implemented by convolution. Simulation experiments demonstrate that for a wide range of imaging systems these small kernels can restore images with fidelity comparable to images restored with the unconstrained Wiener filter     

High-Rate Interpolation of Random Signals From Nonideal Samples

We address the problem of reconstructing a random signal from samples of its filtered version using a given interpolation kernel. In order to reduce the mean squared error (MSE) when using a nonoptimal kernel, we propose a high rate interpolation scheme in which the interpolation grid is finer than the sampling grid. A digital correction system that processes the samples prior to their multiplication with the shifts of the interpolation kernel is developed. This system is constructed such that the reconstructed signal is the linear minimum MSE (LMMSE) estimate of the original signal given its samples. An analytic expression for the MSE as a function of the interpolation rate is provided, which leads to an explicit condition such that the optimal MSE is achieved with the given nonoptimal kernel. Simulations confirm the reduction in MSE with respect to a system with equal sampling and reconstruction rates.      

Flexible hardware-friendly digital architecture for 2-D separable convolution-based scaling

There is not a single scaling technique that suits all kind of images. Final image quality (IQ) depends not only on the scale factor but also on the type of image (photo, CAD, Text...) the user is willing to print or display. Formally, any convolution-based scaling operation can be decomposed in three steps: an anti-aliasing filter, image reconstruction by continuous convolution and resampling to the final grid. Based on this formal framework, we propose a flexible hardware-friendly architecture to perform two-dimensional upscaling and downscaling at low hardware cost. In particular, we propose a discrete convolution engine operating a memory that stores a programmable 2-D-separable interpolation kernel. We also state a technique for optimizing the memory size given the kernel and the scale factor. Finally, we describe a novel flexible filter that overcomes aliasing artifacts regardless of image frequency content. The flexibility provided by the combination of the aforementioned elements allows the user to adjust the interpolation kernel and parameters to each specific type of image for IQ improvement.     

Atom specific multiple kernel dictionary based Sparse Representation Classifier for medium scale image classification

Kernel based Sparse Representation Classifier (KSRC) can classify images with acceptable performance. In addition, Multiple Kernel Learning based SRC (MKL-SRC) computes the weighted sum of multiple kernels in order to construct a unified kernel while the weight of each kernel is calculated as a fixed value in the training phase. In this paper, an MKL-SRC with non-fixed kernel weights for dictionary atoms is proposed. Kernel weights are embedded as new variables to the main KSRC goal function and the resulted optimization problem is solved to find the sparse coefficients and kernel weights simultaneously. As a result, an atom specific multiple kernel dictionary is computed in the training phase which is used by SRC to classify test images. Also, it is proved that the resulting optimization problem is convex and is solvable via common algorithms. The experimental results demonstrate the effectiveness of the proposed approach.     

Rapid gridding reconstruction with a minimal oversampling ratio

Reconstruction of magnetic resonance images from data not falling on a Cartesian grid is a Fourier inversion problem typically solved using convolution interpolation, also known as gridding. Gridding is simple and robust and has parameters, the grid oversampling ratio and the kernel width, that can be used to trade accuracy for computational memory and time reductions. We have found that significant reductions in computation memory and time can be obtained while maintaining high accuracy by using a minimal oversampling ratio, from 1.125 to 1.375, instead of the typically employed grid oversampling ratio of two. When using a minimal oversampling ratio, appropriate design of the convolution kernel is important for maintaining high accuracy. We derive a simple equation for choosing the optimal Kaiser-Bessel convolution kernel for a given oversampling ratio and kernel width. As well, we evaluate the effect of presampling the kernel, a common technique used to reduce the computation time, and find that using linear interpolation between samples adds negligible error with far less samples than is necessary with nearest-neighbor interpolation. We also develop a new method for choosing the optimal presampled kernel. Using a minimal oversampling ratio and presampled kernel, we are able to perform a three-dimensional (3-D) reconstruction in one-eighth the time and requiring one-third the computer memory versus using an oversampling ratio of two and a Kaiser-Bessel convolution kernel, while maintaining the same level of accuracy.     

Building kernels from binary strings for image matching.

In the statistical learning framework, the use of appropriate kernels may be the key for substantial improvement in solving a given problem. In essence, a kernel is a similarity measure between input points satisfying some mathematical requirements and possibly capturing the domain knowledge. In this paper, we focus on kernels for images: we represent the image information content with binary strings and discuss various bitwise manipulations obtained using logical operators and convolution with nonbinary stencils. In the theoretical contribution of our work, we show that histogram intersection is a Mercer's kernel and we determine the modifications under which a similarity measure based on the notion of Hausdorff distance is also a Mercer's kernel. In both cases, we determine explicitly the mapping from input to feature space. The presented experimental results support the relevance of our analysis for developing effective trainable systems.     

卷积神经网络（CNN）训练中卷积核初始化方法研究

本文提出了一种基于样本图像局部模式聚类的卷积核初始化方法,该方法可用于卷积神经网络（Convolutional neural network, CNN）训练中卷积核的初始化。在卷积神经网络中,卷积核的主要作用可看成是利用匹配滤波提取图像中的局部模式,并将其作为后续图像目标识别的特征。为此本文在图像训练集中选取一部分典型的样本图像,在这些图像中抽取与卷积核相同大小的子图作为图像局部模式矢量集合。首先对局部模式子图集合应用拓扑特性进行粗分类,然后对粗分类后的每一子类采用势函数聚类的方法获取样本图像中的典型局部模式子图,构成候选子图模式集,用它们作为CNN的初始卷积核进行训练。实验结果表明,本文方法可以明显加速CNN网络训练初期的收敛速度,同时对最终训练后的网络识别精度也有一定程度的提高。      

基于图像先验和结构特征的盲图像复原算法

提出了一种基于图像先验和图像结构特征的盲图像复原算法,在模糊核未知的情况下,采用一系列离散化的模糊核参数对模糊图像进行非盲去卷积,得到一系列对应的复原图像。同时提出一种复原图像判决准则,对这一系列复原图像进行质量判决,从中得到最优的复原图像。最后在实验部分,通过对图像的测试表明,提出的盲图像复原算法能较准确的得到最优复原图像,复原效果在主观和客观标准上均有良好表现。     

多线阵CCD亚像元成像超分辨率重构技术研究

为实现以多片线阵CCD亚像元成像为基础,提出一种超分辨率重构算法。首先,在高分辨率网格上建立插值模型;然后,辨识插值重构图像在线阵列方向和扫描方向的模糊核,得到整幅图像的模糊核;最后,采用带有Neumman边界条件(BCs)的梯度平滑Richard-Lucy(GSRL)滤波复原算法去除模糊,抑制了振铃效应。实验结果表明,用本文算法重构超分辨率图像的灰度平均梯度(GMG)值较双线性插值法提高了7.63,主观目视清晰、细节丰富;可以实现对多片线阵CCD亚像元成像的超分辨率重构,获取更高的系统分辨率。     

Survey: interpolation methods in medical image processing

Image interpolation techniques often are required in medical imaging for image generation (e.g., discrete back projection for inverse Radon transform) and processing such as compression or resampling. Since the ideal interpolation function spatially is unlimited, several interpolation kernels of finite size have been introduced. This paper compares 1) truncated and windowed sinc; 2) nearest neighbor; 3) linear; 4) quadratic; 5) cubic B-spline; 6) cubic; g) Lagrange; and 7) Gaussian interpolation and approximation techniques with kernel sizes from 1 x 1 up to 8 x 8. The comparison is done by: 1) spatial and Fourier analyses; 2) computational complexity as well as runtime evaluations; and 3) qualitative and quantitative interpolation error determinations for particular interpolation tasks which were taken from common situations in medical image processing. For local and Fourier analyses, a standardized notation is introduced and fundamental properties of interpolators are derived. Successful methods should be direct current (DC)-constant and interpolators rather than DC-inconstant or approximators. Each method's parameters are tuned with respect to those properties. This results in three novel kernels, which are introduced in this paper and proven to be within the best choices for medical image interpolation: the 6 x 6 Blackman-Harris windowed sinc interpolator, and the C2-continuous cubic kernels with N = 6 and N = 8 supporting points. For quantitative error evaluations, a set of 50 direct digital X rays was used. They have been selected arbitrarily from clinical routine. In general, large kernel sizes were found to be superior to small interpolation masks. Except for truncated sinc interpolators, all kernels with N = 6 or larger sizes perform significantly better than N = 2 or N = 3 point methods (p < 0.005). However, the differences within the group of large-sized kernels were not significant. Summarizing the results, the cubic 6 x 6 interpolator with continuous second derivatives, as defined in (24), can be recommended for most common interpolation tasks. It appears to be the fastest six-point kernel to implement computationally. It provides eminent local and Fourier properties, is easy to implement, and has only small errors. The same characteristics apply to B-spline interpolation, but the 6 x 6 cubic avoids the intrinsic border effects produced by the B-spline technique. However, the goal of this study was not to determine an overall best method, but to present a comprehensive catalogue of methods in a uniform terminology, to define general properties and requirements of local techniques, and to enable the reader to select that method which is optimal for his specific application in medical imaging.     

PKCS: A Polynomial Kernel Family With Compact Support for Scale- Space Image Processing

In a scale-space framework, the Gaussian kernel has some properties that make it unique. However, because of its infinite support, exact implementation of this kernel is not possible. To avoid this drawback, there exist two different approaches: approximating the Gaussian kernel by a finite support kernel, or defining new kernels with properties closed to the Gaussian. In this paper, we propose a polynomial kernel family with compact support which overcomes the Gaussian practical drawbacks while preserving a large number of the useful Gaussian properties. The new kernels are not obtained by approximating the Gaussian, though they are derived from it. We show that, for a suitable choice of kernel parameters, this family provides an approximated solution of the diffusion equation and satisfies some other basic constraints of the linear scale-space theory. The construction and properties of the proposed kernel are described, and an application in which handwritten data are extracted from noisy document images is presented.