Demosaicing: image reconstruction from color CCD samples

A simplified color image formation model is used to construct an algorithm for image reconstruction from CCD sensors samples. The proposed method involves two successive steps. The first is motivated by Cok's (1994) template matching technique, while the second step uses steerable inverse diffusion in color. Classical linear signal processing techniques tend to oversmooth the image and result in noticeable color artifacts along edges and sharp features. The question is how should the different color channels support each other to form the best possible reconstruction. Our answer is to let the edges support the color information, and the color channels support the edges, and thereby achieve better perceptual results than those that are bounded by the sampling theoretical limit.     

Minimum norm reconstruction of band-limited function from regularly or irregularly spaced samples

A technique is described for reconstructing a band-limited function from samples given at arbitrary locations on a finite interval. The method coincides with the Nyquist-Shannon interpolation formula when the samples are regularly spaced. Improvements over alternative techniques are illustrated. It is shown that the method gives the best possible reconstruction in the least-squares sense.<>     

The contourlet transform: an efficient directional multiresolution image representation.

The limitations of commonly used separable extensions of one-dimensional transforms, such as the Fourier and wavelet transforms, in capturing the geometry of image edges are well known. In this paper, we pursue a "true" two-dimensional transform that can capture the intrinsic geometrical structure that is key in visual information. The main challenge in exploring geometry in images comes from the discrete nature of the data. Thus, unlike other approaches, such as curvelets, that first develop a transform in the continuous domain and then discretize for sampled data, our approach starts with a discrete-domain construction and then studies its convergence to an expansion in the continuous domain. Specifically, we construct a discrete-domain multiresolution and multidirection expansion using nonseparable filter banks, in much the same way that wavelets were derived from filter banks. This construction results in a flexible multiresolution, local, and directional image expansion using contour segments, and, thus, it is named the contourlet transform. The discrete contourlet transform has a fast iterated filter bank algorithm that requires an order N operations for N-pixel images. Furthermore, we establish a precise link between the developed filter bank and the associated continuous-domain contourlet expansion via a directional multiresolution analysis framework. We show that with parabolic scaling and sufficient directional vanishing moments, contourlets achieve the optimal approximation rate for piecewise smooth functions with discontinuities along twice continuously differentiable curves. Finally, we show some numerical experiments demonstrating the potential of contourlets in several image processing applications. Index Terms-Contourlets, contours, filter banks, geometric image processing, multidirection, multiresolution, sparse representation, wavelets.     

Super-resolution image reconstruction: a technical overview

A new approach toward increasing spatial resolution is required to overcome the limitations of the sensors and optics manufacturing technology. One promising approach is to use signal processing techniques to obtain an high-resolution (HR) image (or sequence) from observed multiple low-resolution (LR) images. Such a resolution enhancement approach has been one of the most active research areas, and it is called super resolution (SR) (or HR) image reconstruction or simply resolution enhancement. In this article, we use the term "SR image reconstruction" to refer to a signal processing approach toward resolution enhancement because the term "super" in "super resolution" represents very well the characteristics of the technique overcoming the inherent resolution limitation of LR imaging systems. The major advantage of the signal processing approach is that it may cost less and the existing LR imaging systems can be still utilized. The SR image reconstruction is proved to be useful in many practical cases where multiple frames of the same scene can be obtained, including medical imaging, satellite imaging, and video applications. The goal of this article is to introduce the concept of SR algorithms to readers who are unfamiliar with this area and to provide a review for experts. To this purpose, we present the technical review of various existing SR methodologies which are often employed. Before presenting the review of existing SR algorithms, we first model the LR image acquisition process.     

Tomographic image reconstruction from a limited number of projections

This paper addresses the reconstruction of a function from its rectangularly sampled Radon transform. After reviewing the Radon transform sampling requirements and describing a convolution/backprojection reconstruction algorithm, two basic issues are addressed. First it is shown that even when the Radon transform of an image is adequately sampled (in the Nyquist sense), the standard convolution/backprojection algorithm may lead to a poor reconstruction unless the Radon transform is first reconstructed to a required set of projections. Secondly, when the Radon transform is undersampled in projections, reconstruction approaches are illustrated which span the range from those which achieve a high uniform-resolution reconstruction with large amounts of aliasing artifacts to those which achieve a low nonuniform-resolution reconstruction with virtually no aliasing artifacts. Reconstruction filters which seek a compromise between those two extremes are defined.     

Exact image reconstruction from a limited number of projections

A new method for the exact reconstruction of any gray-scale image from its projections is proposed. The original image is projected into several view angles and the projection samples are stored in an accumulator array. In order to reconstruct the image, the accumulator array is considered as an accumulation of sinusoidal contributions each one corresponding to a certain pixel of the original image. The proposed method defines conditions for the necessary number of projections and the density of ray samples on the projection axis. These conditions insure that, for each pixel, there is at least one sample in the accumulator array where only this particular pixel contributes. This characteristic projection sample is used during the reconstruction phase to determine the coordinates and the gray-scale value of the corresponding image pixel. A variation of the method is also proposed where the reconstruction is performed using a limited number of projection samples in certain view angles. Specifically, the number of necessary samples equals at most the overall number of pixels in the original image. This approach leads to a significant reduction of memory and processing time requirements since it provides exact image reconstruction using one projection sample per pixel.     

Super-resolution based on fast registration and maximum a posteriori reconstruction.

In this paper, we propose a maximum a posteriori ramework for the super-resolution problem, i.e., reconstructing high-resolution images from shifted, rotated, low-resolution degraded observations. The main contributions of this work are two; first, the use of a new locally adaptive edge preserving prior for the super-resolution problem. Second an efficient two-step reconstruction methodology that includes first an initial registration using only the low-resolution degraded observations. This is followed by a fast iterative algorithm implemented in the discrete Fourier transform domain in which the restoration, interpolation and the registration subtasks of this problem are preformed simultaneously. We present examples with both synthetic and real data that demonstrate the advantages of the proposed framework.     

Three-dimensional face reconstruction from a single image by a coupled RBF network

Reconstruction of a 3-D face model from a single 2-D face image is fundamentally important for face recognition and animation because the 3-D face model is invariant to changes of viewpoint, illumination, background clutter, and occlusions. Given a coupled training set that contains pairs of 2-D faces and the corresponding 3-D faces, we train a novel coupled radial basis function network (C-RBF) to recover the 3-D face model from a single 2-D face image. The C-RBF network explores: 1) the intrinsic representations of 3-D face models and those of 2-D face images; 2) mappings between a 3-D face model and its intrinsic representation; and 3) mappings between a 2-D face image and its intrinsic representation. Since a particular face can be reconstructed by its nearest neighbors, we can assume that the linear combination coefficients for a particular 2-D face image reconstruction are identical to those for the corresponding 3-D face model reconstruction. Therefore, we can reconstruct a 3-D face model by using a single 2-D face image based on the C-RBF network. Extensive experimental results on the BU3D database indicate the effectiveness of the proposed C-RBF network for recovering the 3-D face model from a single 2-D face image.     

Absolute phase image reconstruction: a stochastic nonlinearfiltering approach

This paper formulates and proposes solutions to the problem of estimating/reconstructing the absolute (not simply modulo-2pi) phase of a complex random field from noisy observations of its real and imaginary parts. This problem is representative of a class of important imaging techniques such as interferometric synthetic aperture radar, optical interferometry, magnetic resonance imaging, and diffraction tomography. We follow a Bayesian approach; then, not only a probabilistic model of the observation mechanism, but also prior knowledge concerning the (phase) image to be reconstructed, are needed. We take as prior a nonsymmetrical half plane autoregressive (NSHP AR) Gauss-Markov random field (GMRF). Based on a reduced order state-space formulation of the (linear) NSHP AR model and on the (nonlinear) observation mechanism, a recursive stochastic nonlinear filter is derived, The corresponding estimates are compared with those obtained by the extended Kalman-Bucy filter, a classical linearizing approach to the same problem. A set of examples illustrate the effectiveness of the proposed approach.     

On Bayesian image reconstruction from projections: uniform and nonuniform a priori source information

A method that incorporates a priori uniform or nonuniform source distribution probabilistic information and data fluctuations of a Poisson nature is presented. The source distributions are modeled in terms of a priori source probability density functions. Maximum a posteriori probability solutions, as determined by a system of equations, are given. Interactive Bayesian imaging algorithms for the solutions are derived using an expectation maximization technique. Comparisons of the a priori uniform and nonuniform Bayesian algorithms to the maximum-likelihood algorithm are carried out using computer-generated noise-free and Poisson randomized projections. Improvement in image reconstruction from projections with the Bayesian algorithm is demonstrated. Superior results are obtained using the a priori nonuniform source distribution.     

A Hybrid system for the reconstruction of a smooth function from its samples

The long-standing problem of reconstructing a function from its samples is considered again. Assuming a sequence of oversampled values, a set of appropriate idealized reconstruction filters can be defined, which do not suffer from instability or slow convergence. The realization — a cascade of a nonrecursive digital filter, D/A-converter, and a fixed/analog smoothing filter — demands the design of the digital filter for the increase of the sampling rate. The design of this nonrecursive filter is the purpose of this paper. Approximations in the frequency as well as in the time domain are presented.     

A note on exact image reconstruction from a limited number of projections

In a recent paper in this journal by Kesidis and Papamarkos “A new method for the exact reconstruction of any gray-scale image from its projections is proposed.” In this note we point out that this method is a special case of a well-known approach (peeling) and that it can produce exact reconstructions only under assumptions that are not realistic for practical methods of data collection. Further, we point out that some statements made in the paper regarding disadvantages of the algebraic reconstruction techniques (ART) as compared to the method of the paper are false.     

Estimating the probability of the presence of a signal of interest in multiresolution single- and multiband image denoising.

We develop three novel wavelet domain denoising methods for subband-adaptive, spatially-adaptive and multivalued image denoising. The core of our approach is the estimation of the probability that a given coefficient contains a significant noise-free component, which we call "signal of interest." In this respect, we analyze cases where the probability of signal presence is 1) fixed per subband, 2) conditioned on a local spatial context, and 3) conditioned on information from multiple image bands. All the probabilities are estimated assuming a generalized Laplacian prior for noise-free subband data and additive white Gaussian noise. The results demonstrate that the new subband-adaptive shrinkage function outperforms Bayesian thresholding approaches in terms of mean-squared error. The spatially adaptive version of the proposed method yields better results than the existing spatially adaptive ones of similar and higher complexity. The performance on color and on multispectral images is superior with respect to recent multiband wavelet thresholding.     

A method to perform a fast fourier transform with primitive image transformations.

The Fourier transform is one of the most important transformations in image processing. A major component of this influence comes from the ability to implement it efficiently on a digital computer. This paper describes a new methodology to perform a fast Fourier transform (FFT). This methodology emerges from considerations of the natural physical constraints imposed by image capture devices (camera/eye). The novel aspects of the specific FFT method described include: 1) a bit-wise reversal re-grouping operation of the conventional FFT is replaced by the use of lossless image rotation and scaling and 2) the usual arithmetic operations of complex multiplication are replaced with integer addition. The significance of the FFT presented in this paper is introduced by extending a discrete and finite image algebra, named Spiral Honeycomb Image Algebra (SHIA), to a continuous version, named SHIAC.     

Conductivity image reconstruction from defective data in MREIT: numerical simulation and animal experiment

Magnetic resonance electrical impedance tomography (MREIT) is designed to produce high resolution conductivity images of an electrically conducting subject by injecting current and measuring the longitudinal component, Bz, of the induced magnetic flux density B = (Bx, By, Bz). In MREIT, accurate measurements of Bz are essential in producing correct conductivity images. However, the measured Bz data may contain fundamental defects in local regions where MR magnitude image data are small. These defective Bz data result in completely wrong conductivity values there and also affect the overall accuracy of reconstructed conductivity images. Hence, these defects should be appropriately recovered in order to carry out any MREIT image reconstruction algorithm. This paper proposes a new method of recovering Bz data in defective regions based on its physical properties and neighboring information of Bz. The technique will be indispensable for conductivity imaging in MREIT from animal or human subjects including defective regions such as lungs, bones, and any gas-filled internal organs.     

Pattern tracking and 3-D motion reconstruction of a rigid body from a 2-D image sequence

This paper presents an integrated method to identify an object pattern from an image, and track its movement over a sequence of images. The sequence of images comes from a single perspective video source, which is capturing data from a precalibrated scene. This information is used to reconstruct the scene in three-dimension (3-D) within a virtual environment where a user can interact and manipulate the system. The steps that are performed include the following: i) Identify an object pattern from a two-dimensional perspective video source. The user outlines the region of interest (ROI) in the initial frame; the procedure builds a refined mask of the dominant object within the ROI using the morphological watershed algorithm. ii) The object pattern is tracked between frames using object matching within the mask provided by the previous and next frame, computing the motion parameters. iii) The identified object pattern is matched with a library of shapes to identify a corresponding 3-D object. iv) A virtual environment is created to reconstruct the scene in 3-D using the 3-D object and the motion parameters. This method can be applied to real-life application problems, such as traffic management and material flow congestion analysis.     

Multimodality Bayesian algorithm for image reconstruction inpositron emission tomography: a tissue composition model

The use of anatomical information to improve the quality of reconstructed images in positron emission tomography (PET) has been extensively studied. A common strategy has been to include spatial smoothing within boundaries defined from the anatomical data. The authors present an alternative method for the incorporation of anatomical information into PET image reconstruction, in which they use segmented magnetic resonance (MR) images to assign tissue composition to PET image pixels. The authors model the image as a sum of activities for each tissue type, weighted by the assigned tissue composition. The reconstruction is performed as a maximum a posteriori (MAP) estimation of the activities of each tissue type. Two prior functions, defined for tissue-type activities, are considered. The algorithm is tested in realistic simulations employing a full physical model of the PET scanner     

PDE-based image restoration: a hybrid model and color image denoising.

The paper is concerned with PDE-based image restoration. A new model is introduced by hybridizing a nonconvex variant of the total variation minimization (TVM) and the motion by mean curvature (MMC) in order to deal with the mixture of the impulse and Gaussian noises reliably. We suggest the essentially nondissipative (ENoD) difference schemes for the MMC component to eliminate the impulse noise with a minimum (ideally no) introduction of dissipation. The MMC-TVM hybrid model and the ENoD schemes are applied for both gray-scale and color images. For color image denoising, we consider the chromaticity-brightness decomposition with the chromaticity formulated in the angle domain. An incomplete Crank-Nicolson alternating direction implicit time-stepping procedure is adopted to solve those differential equations efficiently. Numerical experiments have shown that the new hybrid model and the numerical schemes can remove the mixture of the impulse and Gaussian noises, efficiently and reliably, preserving edges quite satisfactorily.     

The reconstruction of a band-limited function and its Fouriertransform from a finite number of samples at arbitrary locations bysingular value decomposition

A method for the stable interpolation of a bandlimited function known at sample instants with arbitrary locations in the presence of noise is given. Singular value decomposition is used to provide a series expansion that, in contrast to the method of sampling functions, permits simple identification of vectors in the minimum-norm space poorly represented in the sample values. Three methods, Miller regularization, least squares estimation, and maximum a posteriori estimation, are given for obtaining regularized reconstructions when noise is present. The singular value decomposition (SVD) method is used to interrelate these methods. Examples illustrating the technique are given     

Three-dimensional (3-D) reconstruction of the spine from a single X-ray image and prior vertebra models

The lateral bending test is routinely used by clinicians for the preoperative assessment of spinal mobility. The evaluation of bending motion is usually based on the qualitative analysis of a two-dimensional (2-D) antero-posterior X-ray image. The aim of this paper is to introduce a novel three-dimensional (3-D) reconstruction technique that is a prerequisite for the quantitative 3-D analysis of lateral bending motion. An algorithm was developed for the 3-D reconstruction of the spine from a single X-ray image. The X-ray is calibrated using a small calibration object and an explicit calibration algorithm. The information contained in the single X-ray is completed by registering a priori 3-D geometric models of individual vertebrae. Part of the error yielded by the 3-D/2-D registration is corrected by a vertebral alignment constraint that aims to minimize intervertebral dislocations. Three-dimensional models of 15 different scoliosis patients, obtained from a standard stereo-radiographic 3-D reconstruction, were used in simulation and validation experiments. Experimental results show that the new method is robust and accurate. With pessimistic levels of simulated noise, the average root mean square reconstruction error is 2.89 mm, which is appropriate for common clinical applications.