High-resolution images from low-resolution compressed video

Surveys the field of super resolution (SR) processing for compressed video. The introduction of motion vectors, compression noise, and additional redundancies within the image sequence makes this problem fertile ground for novel processing methods. In conducting this survey, though, we develop and present all techniques within the Bayesian framework. This adds consistency to the presentation and facilitates comparison between the different methods. The article is organized as follows. We define the acquisition system utilized by the surveyed procedures. Then we formulate the HR problem within the Bayesian framework and survey models for the acquisition and compression systems. This requires consideration of both the motion vectors and transform coefficients within the compressed bit stream. We survey models for the original HR image intensities and displacement values. We discuss solutions for the SR problem and provide examples of several approaches.     

A VQ-based blind image restoration algorithm

Learning-based algorithms for image restoration and blind image restoration are proposed. Such algorithms deviate from the traditional approaches in this area, by utilizing priors that are learned from similar images. Original images and their degraded versions by the known degradation operator (restoration problem) are utilized for designing the VQ codebooks. The codevectors are designed using the blurred images. For each such vector, the high frequency information obtained from the original images is also available. During restoration, the high frequency information of a given degraded image is estimated from its low frequency information based on the codebooks. For the blind restoration problem, a number of codebooks are designed corresponding to various versions of the blurring function. Given a noisy and blurred image, one of the codebooks is chosen based on a similarity measure, therefore providing the identification of the blur. To make the restoration process computationally efficient, the principal component analysis (PCA) and VQ-nearest neighbor approaches are utilized. Simulation results are presented to demonstrate the effectiveness of the proposed algorithms.     

Bayesian multichannel image restoration using compound Gauss-Markov random fields

We develop a multichannel image restoration algorithm using compound Gauss-Markov random fields (CGMRF) models. The line process in the CGMRF allows the channels to share important information regarding the objects present in the scene. In order to estimate the underlying multichannel image, two new iterative algorithms are presented and their convergence is established. They can be considered as extensions of the classical simulated annealing and iterative conditional methods. Experimental results with color images demonstrate the effectiveness of the proposed approaches.     

Audio-visual interaction in multimodal communication

Multimedia signal processing is more than simply “putting together” text, audio, images, and video. It is the integration and interaction among these different media that creates new systems and new research challenges and opportunities. In multimodal communication where speech is involved, audio-visual interaction is particularly significant     

Motion estimation of skeletonized angiographic images using elastic registration

An approach for estimating the motion of arteries in digital angiographic image sequences is proposed. Binary skeleton images are registered using an elastic registration algorithm in order to estimate the motion of the corresponding arteries. This algorithm operates recursively on the skeleton images by considering an autoregressive (AR) model of the deformation in conjunction with a dynamic programming (DP) algorithm. The AR model is used at the pixel level and provides a suitable cost function to DP through the innovation process. In addition, a moving average (MA) model for the motion of the entire skeleton is used in combination with the local AR model for improved registration results. The performance of this motion estimation method is demonstrated on simulated and real digital angiographic image sequences. It is shown that motion estimation using elastic registration of skeletons is very successful especially with low contrast and noisy angiographic images.     

Parameter estimation in Bayesian reconstruction of SPECT images: An aid in nuclear medicine diagnosis

Despite the adequacy of Bayesian methods to reconstruct nuclear medicine SPECT (single‐photon emission computed tomography) images, they are rarely used in everyday medical practice. This is primarily because of their computational cost and the need to appropriately select the prior model hyperparameters. We propose a simple procedure for the estimation of these hyperparameters and the reconstruction of the original image and test the procedure on both synthetic and real SPECT images. The experimental results demonstrate that the proposed hyperparameter estimation method produces satisfactory reconstructions. Although we have used generalized Gaussian Markov random fields (GGMRF) as prior models, the proposed estimation method can be applied to any priors with convex potential and tractable partition function with respect to the scale hyperparameter. © 2004 Wiley Periodicals, Inc. Int J Imaging Syst Technol 14, 21–27, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.20003     

An HMM-based speech-to-video synthesizer

Emerging broadband communication systems promise a future of multimedia telephony, e.g. the addition of visual information to telephone conversations. It is useful to consider the problem of generating the critical information useful for speechreading, based on existing narrowband communications systems used for speech. This paper focuses on the problem of synthesizing visual articulatory movements given the acoustic speech signal. In this application, the acoustic speech signal is analyzed and the corresponding articulatory movements are synthesized for speechreading. This paper describes a hidden Markov model (HMM)-based visual speech synthesizer. The key elements in the application of HMMs to this problem are the decomposition of the overall modeling task into key stages and the judicious determination of the observation vector's components for each stage. The main contribution of this paper is a novel correlation HMM model that is able to integrate independently trained acoustic and visual HMMs for speech-to-visual synthesis. This model allows increased flexibility in choosing model topologies for the acoustic and visual HMMs. Moreover the propose model reduces the amount of training data compared to early integration modeling techniques. Results from objective experiments analysis show that the propose approach can reduce time alignment errors by 37.4% compared to conventional temporal scaling method. Furthermore, subjective results indicated that the purpose model can increase speech understanding.     

A Bayesian approach for the estimation and transmission ofregularization parameters for reducing blocking artifacts

With block-based compression approaches for both still images and sequences of images annoying blocking artifacts are exhibited, primarily at high compression ratios. They are due to the independent processing (quantization) of the block transformed values of the intensity or the displaced frame difference. We propose the application of the hierarchical Bayesian paradigm to the reconstruction of block discrete cosine transform (BDCT) compressed images and the estimation of the required parameters. We derive expressions for the iterative evaluation of these parameters applying the evidence analysis within the hierarchical Bayesian paradigm. The proposed method allows for the combination of parameters estimated at the coder and decoder. The performance of the proposed algorithms is demonstrated experimentally.