Resolution enhancement of monochrome and color video using motioncompensation

We propose an iterative algorithm for enhancing the resolution of monochrome and color image sequences. Various approaches toward motion estimation are investigated and compared. Improving the spatial resolution of an image sequence critically depends upon the accuracy of the motion estimator. The problem is complicated by the fact that the motion field is prone to significant errors since the original high-resolution images are not available. Improved motion estimates may be obtained by using a more robust and accurate motion estimator, such as a pel-recursive scheme instead of block matching, in processing color image sequences, there is the added advantage of having more flexibility in how the final motion estimates are obtained, and further improvement in the accuracy of the motion field is therefore possible. This is because there are three different intensity fields (channels) conveying the same motion information. In this paper, the choice of which motion estimator to use versus how the final estimates are obtained is weighed to see which issue is more critical in improving the estimated high-resolution sequences. Toward this end, an iterative algorithm is proposed, and two sets of experiments are presented. First, several different experiments using the same motion estimator but three different data fusion approaches to merge the individual motion fields were performed. Second, estimated high-resolution images using the block matching estimator were compared to those obtained by employing a pel-recursive scheme. Experiments were performed on a real color image sequence, and performance was measured by the peak signal to noise ratio (PSNR).     

Audio-Visual Biometrics

Biometric characteristics can be utilized in order to enable reliable and robust-to-impostor-attacks person recognition. Speaker recognition technology is commonly utilized in various systems enabling natural human computer interaction. The majority of the speaker recognition systems rely only on acoustic information, ignoring the visual modality. However, visual information conveys correlated and complimentary information to the audio information and its integration into a recognition system can potentially increase the system's performance, especially in the presence of adverse acoustic conditions. Acoustic and visual biometric signals, such as the person's voice and face, can be obtained using unobtrusive and user-friendly procedures and low-cost sensors. Developing unobtrusive biometric systems makes biometric technology more socially acceptable and accelerates its integration into every day life. In this paper, we describe the main components of audio-visual biometric systems, review existing systems and their performance, and discuss future research and development directions in this area     

Image and video compression algorithms based on recovery techniquesusing mean field annealing

Image and video coding algorithms have found a number of applications ranging from video telephony on the public switched telephone networks (PSTN) to HDTV. However, as the bit rate is lowered, most of the existing techniques, as well as current standards, such as JPEG, H. 261, and MPEG-1 produce highly visible degradations in the reconstructed images primarily due to the information loss caused by the quantization process. In this paper, we propose an iterative technique to reduce the unwanted degradations, such as blocking and mosquito artifacts while keeping the necessary detail present in the original image. The proposed technique makes use of a priori information about the original image through a nonstationary Gauss-Markov model. Utilizing this model, a maximum a posteriori (MAP) estimate is obtained iteratively using mean field annealing. The fidelity to the data is preserved by projecting the image onto a constraint set defined by the quantizer at each iteration. The proposed solution represents an implementation of a paradigm we advocate, according to which the decoder is not simply undoing the operations performed by the encoder, but instead it solves an estimation problem based on the available bitstream and any prior knowledge about the source image. The performance of the proposed algorithm was tested on a JPEG, as well as on an H.261-type video codec. It is shown to be effective in removing the coding artifacts present in low bit rate compression     

Variational Bayesian Image Restoration With a Product of Spatially Weighted Total Variation Image Priors

In this paper, a new image prior is introduced and used in image restoration. This prior is based on products of spatially weighted total variations (TV). These spatial weights provide this prior with the flexibility to better capture local image features than previous TV based priors. Bayesian inference is used for image restoration with this prior via the variational approximation. The proposed restoration algorithm is fully automatic in the sense that all necessary parameters are estimated from the data and is faster than previous similar algorithms. Numerical experiments are shown which demonstrate that image restoration based on this prior compares favorably with previous state-of-the-art restoration algorithms.      

Operationally optimal vertex-based shape coding

In this article, we address the issue of operationally optimal shape encoding, which is a step in the direction of globally optimal resource allocation in object-oriented video. After an overview of shape-based coding and algorithms, we define the problem mathematically, introduce the necessary notation, and then present the basic idea behind the proposed algorithms. We then discuss the constraints imposed on the code used to encode the approximation. We then introduce a definition of distortion that fits into the proposed framework and introduce the directed acyclic graph (DAG) formulation of the problem, which results in a fast solution approach. We also show how the DAG algorithm can be used to find the approximation with the minimum-maximum segment distortion for a given rate as well as to find the approximation with the smallest total distortion for a given rate. We then present experimental results and point out directions for future research     

An optimal polygonal boundary encoding scheme in the ratedistortion sense

In this paper, we present fast and efficient methods for the lossy encoding of object boundaries that are given as eight-connect chain codes. We approximate the boundary by a polygon, and consider the problem of finding the polygon which leads to the smallest distortion for a given number of bits. We also address the dual problem of finding the polygon which leads to the smallest bit rate for a given distortion. We consider two different classes of distortion measures. The first class is based on the maximum operator and the second class is based on the summation operator. For the first class, we derive a fast and optimal scheme that is based on a shortest path algorithm for a weighted directed acyclic graph. For the second class we propose a solution approach that is based on the Lagrange multiplier method, which uses the above-mentioned shortest path algorithm. Since the Lagrange multiplier method can only find solutions on the convex hull of the operational rate distortion function, we also propose a tree-pruning-based algorithm that can find all the optimal solutions. Finally, we present results of the proposed schemes using objects from the Miss America sequence.     

MPEG-4 and rate-distortion-based shape-coding techniques

We address the problem of the efficient encoding of object boundaries. This problem is becoming increasingly important in applications such as content-based storage and retrieval, studio and television postproduction, and mobile multimedia applications. The MPEG-4 visual standard will allow the transmission of arbitrarily shaped video objects. The techniques developed for shape coding within the MPEG-4 standardization effort are described and compared first. A framework for the representation of shapes using their contours is presented next. Such representations are achieved using curves of various orders, and they are optimal in the rate-distortion sense. Finally, conclusions are drawn     

Simultaneous multichannel image restoration and estimation of theregularization parameters

In this correspondence, a constrained least-squares multichannel image restoration approach is proposed, in which no prior knowledge of the noise variance at each channel or the degree of smoothness of the original image is required. The regularization functional for each channel is determined by incorporating both within-channel and cross-channel information. It is shown that the proposed smoothing functional has a global minimizer.     

Bayesian resolution enhancement of compressed video

Super-resolution algorithms recover high-frequency information from a sequence of low-resolution observations. In this paper, we consider the impact of video compression on the super-resolution task. Hybrid motion-compensation and transform coding schemes are the focus, as these methods provide observations of the underlying displacement values as well as a variable noise process. We utilize the Bayesian framework to incorporate this information and fuse the super-resolution and post-processing problems. A tractable solution is defined, and relationships between algorithm parameters and information in the compressed bitstream are established. The association between resolution recovery and compression ratio is also explored. Simulations illustrate the performance of the procedure with both synthetic and nonsynthetic sequences.