Multi-objective super resolution: concepts and examples

Described methods for simultaneously generating the super-resolved depth map and the image from LR observations. Structural information is embedded within the observations and, through the two formulations of DFD and SFS problems, we were able to generate the super-resolved images and the structures. The first method described here avoids correspondence and warping problems inherent in current SR techniques involving the motion cue in the LR observations and uses a more natural depth-related defocus as a natural cue in real aperture imaging. The second method, while again avoiding the correspondence problems, also demonstrates the usefulness of the generalized interpolation scheme leading to more flexibility in the final SR image, in the sense that the LR image can be viewed at SR with an arbitrary light source position. The quality of the super-resolved depth and intensity maps has been found to be quite good. The MAP-MRF framework that was used in both methods models both the surface normal and the intensity field as separate MRFs, and this helps in regularizing the solution.     

Efficient local stereo matching algorithm based on fast gradient domain guided image filtering

Guided image filtering (GIF) based cost aggregation or disparity refinement stereo matching algorithms are studied extensively owing to the edge-aware preserved smoothing property. However, GIF suffers from halo artifacts in sharp edges and shows high computational costs on high-resolution images. The performance of GIF in stereo matching would be limited by the above two defects. To solve these problems, a novel fast gradient domain guided image filtering (F-GDGIF) is proposed. To be specific, halo artifacts are effectively alleviated by incorporating an efficient multi-scale edge-aware weighting into GIF. With this multi-scale weighting, edges can be preserved much better. In addition, high computational costs are cut down by sub-sampling strategy, which decreases the computational complexity from O(N) to O(N/s²) (s: sub-sampling ratio) To verify the effectiveness of the algorithm, F-GDGIF is applied to cost aggregation and disparity refinement in stereo matching algorithms respectively. Experiments on the Middlebury evaluation benchmark demonstrate that F-GDGIF based stereo matching method can generate more accuracy disparity maps with low computational cost compared to other GIF based methods.     

Image annotation by semi-supervised cross-domain learning with group sparsity

With the explosive growth of multimedia data in the web, multi-label image annotation has been attracted more and more attention. Although the amount of available data is large and growing, the number of labeled data is quite small. This paper proposes an approach to utilize both unlabeled data in target domain and labeled data in auxiliary domain to boost the performance of image annotation. Moreover, since different kinds of heterogeneous features in images have different intrinsic discriminative power for image understanding, group sparsity is introduced in our approach to effectively utilize those heterogeneous visual features with data of target and auxiliary domains. We call this approach semi-supervised cross-domain learning with group sparsity (S²CLGS). The strength of the proposed S²CLGS method for multi-label image annotation is to integrate semi-supervised discriminant analysis, cross-domain learning and sparse coding together. Experiments demonstrate the effectiveness of S²CLGS in comparison with other image annotation algorithms.     

A novel image restoration scheme based on structured side information and its application to image watermarking

This paper presents a new image restoration method based on a linear optimization model which restores part of the image from structured side information (SSI). The SSI can be transmitted to the receiver or embedded into the image itself by a digital watermarking technique. In this paper we focus on a special type of SSI for digital watermarking where the SSI is composed of mean values of 4×4 image blocks which can be used to restore manipulated blocks. Different from existing image restoration methods for similar types of SSI, the proposed method minimizes image discontinuity according to a relaxed definition of smoothness based on a 3×3 averaging filter of four adjacent pixel value differences, and the objective function of the optimization model has a second regularization term corresponding to a second-order smoothness criterion. Our experiments on 100 test images showed that given complete information of the SSI, the proposed image restoration technique can outperform the state-of-the-art model based on a simple linear optimization model by around 2 dB in terms of an average Peak Signal-to-Noise Ratio (PSNR) value and around 0.04 in terms of a Structural Similarity Index (SSIM) value. We also tested the robustness of the image restoration method when it is applied to a self-restoration watermarking scheme and the experimental results showed that it is moderately robust to errors in SSI (which is embedded as a watermark) caused by JPEG compression (the average PSNR value remains above 16.5 dB even when the JPEG QF is 50), additive Gaussian white noises (the average PSNR value is approximately 18.4 dB when the noise variance σ² is 10) and image rescaling assuming the original image size is known at the receiver side (e.g. the average PSNR value is approximately 19.6 dB when the scaling ratio is 1.4).     

Parallel analog to digital conversion with speed/precision trade-off

A new method for multichannel A/D-conversion is presented with a typical application being parallel read-out from a semiconductor image sensor. Each analog input signal is fed to a pair of comparators, the thresholds AB of which are delivered in parallel to all channels. With a proper sequence of A- and B-levels, digital outputs can be produced bit-serially for all channels in parallel. The problems of image sensor design is greatly alleviated since the bandwidth of each output channel is only a fraction of the bandwidth of a conventional video signal. The main virtue of this method is that precision in the digital output can be traded for speed. For a 256 comparator system the complexity and speed potential is the same as for an 8-bit flash converter. However, the new converter doubles its speed for each one-bit decrease in precision. Furthermore, all kinds of nonlinear compression and transformation of the input signal range are under instantaneous and programmable control. It is believed that the new converter opens the doors to more efficient image processing e.g. in robot vision. Flexibility and adaptation can be extended from high level parts of the systems all the way down to the image pick-up level.     

PLDD: Point-lines distance distribution for detection of arbitrary triangles,regular polygons and circles

In this paper, a general framework is presented for detection of arbitrary triangles, regular polygons, and circles, which is inspired by the common geometric property that the incenter of the shape is equidistant to the tangential lines of the contour points. The idea of point-lines distance distribution (PLDD) is introduced to compute the shape energy of each pixel. Then, shape centers can be exacted from the produced PLDD map, and shape radii are obtained simultaneously based on the distance distribution of the shape center. The shape candidates are thus determined and represented with three independent characteristics: shape center, shape radius, and contour points. Finally, distinguish different types of the shape from shape candidates using shape contour points information. Compared with exiting methods, the PLDD based method detects the shapes mainly using the inherent information of edge points, such as distance, and it is simple and general. Comparative experiments both on synthetic and natural images with the state of the art also prove that the PLDD based method performs more robustly and accurately with the maximal time complexity O(n²) at the worst condition.     

Efficient Hadamard transformation of large images

A method for performing the two-dimensional Hadamard transformation is presented. It is based on a fast Hadamard algorithm as used in digital data processing. For the processing of an image consisting of N² pixels, 2 log₂N identical steps are necessary. The operation for a specific step is subdivided into two space-invariant suboperations and one multiplication. Space-invariance is highly desirable for the parallel optical implementation. The required space-bandwidth product for the algorithm is 2N². Hence large images (e.g. 1000² pixels) can be processed in parallel.     

An adaptive two phase blind image deconvolution algorithm for an iterative regularization model

This paper proposes a blind image deconvolution method which consists of two sequential phases, i.e., blur kernel estimation and image restoration. In the first phase, we adopt the L₀-norm of image gradients and total variation (TV) to regularize the latent image and blur kernel, respectively. Then we design an alternating optimization algorithm which jointly incorporates the estimation of intermediately restored image, blur kernel and regularization parameters into account. In the second phase, we propose to take the mixture of L₀-norm of image gradients and TV to regularize the latent image, and design an efficient non-blind deconvolution algorithm to achieve the restored image. Experimental results on both a benchmark image dataset and real-world blurred images show that the proposed method can effectively restore image details while suppress noise and ringing artifacts, the result is of high quality which is competitive with some state of the art methods.     

Image selective restoration using multi-scale variational decomposition

In this paper we propose a multi-scale variational decomposition model for image selective restoration. Firstly, we introduce a single-parameter (BV, G, L²) variational decomposition functional and theoretically analyze the relationship between the parameter and the scale of image features. And then, by replacing the fixed scale parameter with a varying sequence in the single-parameter decomposition functional, we obtain the multi-scale variational decomposition which can decompose the input image into a series of image slices of different scales. Furthermore, we show some properties and prove the convergence of the multi-scale decomposition. Finally, we introduce an alternating and iterative method based on Chambolle’s projection algorithm to numerically solve the multi-scale variational decomposition model. Experiments are conducted on both synthetic and real images to demonstrate the effectiveness of the proposed multi-scale variational decomposition. In addition, we use the multi-scale variational decomposition to achieve image selective restoration, and compare it with several state-of-the-art models in denoising application. The numerical results show that our model has better performance in terms of PSNR and SSIM indexes.     

Stain removal in 2D images with globally varying textures

In this paper, we deal with the problem of removing stains in an image that may contain globally varying textures. In general, Image inpainting and texture synthesis are two possible techniques that may be used to address this issue; however, each has its limitation. In this work, we propose an approach that helps to address this problem, especially when the target image portions to be repaired may consist of globally varying textures, that is, textures that are not stationary, but vary globally across images due to lighting conditions or composing materials. We have developed methods to first detect regions with stains and then remove the stains to obtain a newer look for the input image. Results are shown and compared with those of others if applicable to prove the effectiveness of proposed approaches.     

A new methodology for quantifying the multi-scale similarity of images

Metrology of devices becoming more and more sophisticated, the collected information is subsequently still increasing. The characteristics of an engineering surface can often be recorded as an image. To compare the characteristics of two different engineering surfaces X and Y tailored with different process parameters, to determine process parameters that have to be controlled to produce a surface with desired properties or to quantify the relevance of a post image treatment for characterising a particular surface property, a practical problem of major interest is therefore to answer the question “are images related to surfaces X and Y similar at all the length scales?”. An original method, based on recent information theory assumptions and on the multi-fractal formalism, is proposed to quantify the degree of similarity of a set of images at all the length scales. The relevance of this method for characterising the morphological textures of surfaces was developed on simulated images generated by means of a 3D fractal function simulating an abrasion process.     

Sparse intrinsic decomposition and applications

This paper proposes an intrinsic decomposition method from a single RGB-D image. To remedy the highly ill-conditioned problem, the reflectance component is regularized by a sparsity term, which is weighted by a bilateral kernel to exploit non-local structural correlation. As shading images are piece-wise smooth and have sparse gradient fields, the sparse-induced ${\ell }_1$-norm is used to regularize the finite difference of the direct irradiance component, which is the most dominant sub-component of shading and describes the light directly received by the surfaces of the objects from the light source. To derive an efficient algorithm, the proposed model is transformed into an unconstrained minimization of the augmented Lagrangian function, which is then optimized via the alternating direction method. The stability of the proposed method with respect to parameter perturbation and its robustness to noise are investigated by experiments. Quantitative and qualitative evaluation demonstrates that our method has better performance than state-of-the-art methods. Our method can also achieve intrinsic decomposition from a single color image by integrating existed depth estimation methods. We also present a depth refinement method based on our intrinsic decomposition method, which obtains more geometry details without texture artifacts. Other application, e.g., texture editing, also demonstrates the effectiveness of our method.     

An augmented Lagrangian approach to general dictionary learning for image denoising

This paper presents an augmented Lagrangian (AL) based method for designing of overcomplete dictionaries for sparse representation with general l_q-data fidelity term (q ? 2). In the proposed method, the dictionary is updated via a simple gradient descent method after each inner minimization step of the AL scheme. Besides, a modified Iterated Shrinkage/Thresholding Algorithm is employed to accelerate the sparse coding stage of the algorithm. We reveal that the dictionary update strategy of the proposed method is different from most of existing methods because the learned dictionaries become more and more complex regularly. An advantage of the iterated refinement methodology is that it makes the method less dependent on the initial dictionary. Experimental results on real image for Gaussian noise removal (q = 2) and impulse noise removal (q = 1) consistently demonstrate that the proposed approach can efficiently remove the noise while maintaining high image quality.     

Field dependences of the conductance of disordered metal/oxide/semiconductor silicon structures with an inversion p-type channel

An experimental procedure for studying and analyzing conductance G of the inversion channel as a function of transverse voltage V _g (the field effect) and longitudinal voltage V _d is developed for metal/oxide/semiconductor (MOS) structures. The presence of a quasi-plateau in dependence G(V _g) at G ≈ 2e ²/h and a dip in dependence G(V _d) at |V _d | ≤ 100 mV with a minimum at V _g = 0 are demonstrated by using the example of mesoscopic Si-MOS structures with an inversion p-type channel and an increased concentration of embedded charges at 77 K. The effective energy parameters of quantum contacts formed in the saddle-point regions of the fluctuation potential are determined. The experimental accuracy obtained allows the use of regularizing algorithms for consideration of the effect of the final temperature.     

Determination of the energy distribution of interface traps in MIS systems using non-steady-state techniques

Experimental techniques are described for determining the energy distribution of interface traps at the semiconductor-insulator interface of MIS devices. The device used here was an MNOS capacitor in which the semiconductor was n-type. The first technique which is described is that of measuring the thermally stimulated currents. The method consists of biasing the capacitor into the accumulation mode at a low temperature thereby filling the traps at the semiconductor oxide interface. The device is then biased into the deep-depletion mode in which state the traps remain filled because the temperature is too low to allow the electrons to be thermally excited out of the traps. The temperature of the device is then raised at a uniform rate, and the current associated with the release of electrons from the trap is monitored. The shape of the I?T characteristic is a direct image of the interface trap distribution is a broad peak with a maximum at 0·35 eV below the bottom of the conduction band, and of height approximately 6 × 10¹³ cm^?2eV^?1. The experiments were carried out at two heating rates (0·1°K/sec and 0·01°K/sec), and the trap densities so obtained were identical.The second method consists of biasing the device into the accumulation mode at a fixed temperature thereby filling the traps at the silicon-silicon oxide interface. It is then short-circuited and the non-steady state transient current associated with the release of electrons from the interface traps is monitored. The energy distribution of the interface traps in the upper half of the forbidden gap is shown to be readily obtained from the transient currents, and is found to be identical to that obtained using the thermal technique.     

An auxiliary theorem for stability analysis in the presence of interval-valued parameters

A theorem which helps to determine the boundary of the image of an interval inR ⁿ under a differentiable mapping,F:R ⁿ C, is expounded. Examples illustrating the application of the theorem are given.     

A fully automatic one-scan adaptive zooming algorithm for color images

We present an interpolation algorithm for adaptive color image zooming. The algorithm produces the magnified image in one scan of the input image, and is fully automatic since does not involve any a priori fixed threshold. Given any integer zooming factor n, each pixel of the input image generates an n×n block of pixels in the zoomed image. For the currently visited pixel of the input image, the pixels of its associated block are first assigned tentative values, which are then adaptively updated before building the next block. The method is suggested for RGB images, but can equally be employed in other color spaces. Peak signal to noise ratio (PSNR) and Structural SIMilarity (SSIM) are used to evaluate the performance of the algorithm.     

Image coarsening by using space-filling curve for decomposition-based image enhancement

We propose a novel space-filling curve based image coarsening method, which automatically extracts a base-layer from an input image while still preserving its structural context, meaningful details, et cetera. In the proposed method, specifically, a one-dimensional edge-preserving smoothing filter, which is called a vector ε-filter, is applied to an input image along a space-filling curve. In this regard, the space-filling curve is constructed by using a minimum spanning tree which extracts the structural context of the input image. This novel image coarsening approach is completely different from all conventional approaches employing any kind of two-dimensional filter window. Furthermore, this coarsening method can effectively produce an aggregation of texture details as well as enhance sharp edges, while preserving structural contexts such as thin lines and sharp corners. The main benefit of the coarsened image by the proposed method is its suitability for extracting fine features of an input image for decomposition-based image enhancement. In this paper, the structural-context-preserving image coarsening capability of the proposed method is verified by some results from experiments and examples. Then we show our new method’s characteristics in practical application to decomposition-based image enhancement by using some other examples.     

Restoration of multispectral images by the gradient reconstruction method and estimation of the blur parameters on the basis of the multipurpose matching model

In this study, an original method for restoration of multispectral images by means of gradient reconstruction is proposed. The method uses a simple relationship between the gradient at the neighboring points in the distorted image and the gradient at distant points in the initial image. The result of restoration by the proposed algorithm excels the result obtained by the standard method based on the Wiener filtering. A new method for estimation of the parameters of the distorting motion-blur operator is also proposed. In this case, the blurred image is considered as a superposition of M shifted original images, and the autocorrelation convolution of the distorted image can be represented as a linear combination of M ² mutual convolutions of several identical shifted images. Thus, the autocorrelation function of the distorted image is a straight line passing through the center, and the direction of this line and its length coincides with the parameters of the distorting operator. As compared to the best present-day algorithms, the proposed method exhibits higher accuracy of parameter estimation. In addition, computation of parameters with the use of this method takes much less time than with the use of popular estimation algorithms based on the Radon transform.