Shape error concealment using Hermite splines

The introduction of video objects (VOs) is one of the innovations of MPEG-4. The alpha-plane of a VO defines its shape at a given instance in time and hence determines the boundary of its texture. In packet-based networks, shape, motion, and texture are subject to loss. While there has been considerable attention paid to the concealment of texture and motion errors, little has been done in the field of shape error concealment. In this paper, we propose a post-processing shape error-concealment technique that uses geometric boundary information of the received alpha-plane. Second-order Hermite splines are used to model the received boundary in the neighboring blocks, while third order Hermite splines are used to model the missing boundary. The velocities of these splines are matched at the boundary point closest to the missing block. There exists the possibility of multiple concealing splines per group of lost boundary parts. Therefore, we draw every concealment spline combination that does not self-intersect and keep all possible results until the end. At the end, we select the concealment solution that results in one closed boundary. Experimental results demonstrating the performance of the proposed method and comparisons with prior proposed methods are presented.     

Joint source-channel coding for motion-compensated DCT-based SNR scalable video

In this paper, we develop an approach toward joint source-channel coding for motion-compensated DCT-based scalable video coding and transmission. A framework for the optimal selection of the source and channel coding rates over all scalable layers is presented such that the overall distortion is minimized. The algorithm utilizes universal rate distortion characteristics which are obtained experimentally and show the sensitivity of the source encoder and decoder to channel errors. The proposed algorithm allocates the available bit rate between scalable layers and, within each layer, between source and channel coding. We present the results of this rate allocation algorithm for video transmission over a wireless channel using the H.263 Version 2 signal-to-noise ratio (SNR) scalable codec for source coding and rate-compatible punctured convolutional (RCPC) codes for channel coding. We discuss the performance of the algorithm with respect to the channel conditions, coding methodologies, layer rates, and number of layers.     

VAPOR: variance-aware per-pixel optimal resource allocation.

Characterizing the video quality seen by an end-user is a critical component of any video transmission system. In packet-based communication systems, such as wireless channels or the Internet, packet delivery is not guaranteed. Therefore, from the point-of-view of the transmitter, the distortion at the receiver is a random variable. Traditional approaches have primarily focused on minimizing the expected value of the end-to-end distortion. This paper explores the benefits of accounting for not only the mean, but also the variance of the end-to-end distortion when allocating limited source and channel resources. By accounting for the variance of the distortion, the proposed approach increases the reliability of the system by making it more likely that what the end-user sees, closely resembles the mean end-to-end distortion calculated at the transmitter. Experimental results demonstrate that variance-aware resource allocation can help limit error propagation and is more robust to channel-mismatch than approaches whose goal is to strictly minimize the expected distortion.     

Motion compensated shape error concealment.

The introduction of Video Objects (VOs) is one of the innovations of MPEG-4. The alpha-plane of a VO defines its shape at a given instance in time and hence determines the boundary of its texture. In packet-based networks, shape, motion, and texture are subject to loss. While there has been considerable attention paid to the concealment of texture and motion errors, little has been done in the field of shape error concealment. In this paper we propose a post-processing shape error concealment technique that uses the motion compensated boundary information of the previously received alpha-plane. The proposed approach is based on matching received boundary segments in the current frame to the boundary in the previous frame. This matching is achieved by finding a maximally smooth motion vector field. After the current boundary segments are matched to the previous boundary, the missing boundary pieces are reconstructed by motion compensation. Experimental results demonstrating the performance of the proposed motion compensated shape error concealment method, and comparing it with the previously proposed weighted side matching method are presented.     

Spatially adaptive wavelet-based multiscale image restoration

In this paper, we present a new spatially adaptive approach to the restoration of noisy blurred images, which is particularly effective at producing sharp deconvolution while suppressing the noise in the flat regions of an image. This is accomplished through a multiscale Kalman smoothing filter applied to a prefiltered observed image in the discrete, separable, 2-D wavelet domain. The prefiltering step involves constrained least-squares filtering based on optimal choices for the regularization parameter. This leads to a reduction in the support of the required state vectors of the multiscale restoration filter in the wavelet domain and improvement in the computational efficiency of the multiscale filter. The proposed method has the benefit that the majority of the regularization, or noise suppression, of the restoration is accomplished by the efficient multiscale filtering of wavelet detail coefficients ordered on quadtrees. Not only does this lead to potential parallel implementation schemes, but it permits adaptivity to the local edge information in the image. In particular, this method changes filter parameters depending on scale, local signal-to-noise ratio (SNR), and orientation. Because the wavelet detail coefficients are a manifestation of the multiscale edge information in an image, this algorithm may be viewed as an "edge-adaptive" multiscale restoration approach.     

General choice of the regularization functional in regularizedimage restoration

The determination of the regularization parameter is an important issue in regularized image restoration, since it controls the trade-off between fidelity to the data and smoothness of the solution. A number of approaches have been developed in determining this parameter. In this paper, a new paradigm is adopted, according to which the required prior information is extracted from the available data at the previous iteration step, i.e., the partially restored image at each step. We propose the use of a regularization functional instead of a constant regularization parameter. The properties such a regularization functional should satisfy are investigated, and two specific forms of it are proposed. An iterative algorithm is proposed for obtaining a restored image. The regularization functional is defined in terms of the restored image at each iteration step, therefore allowing for the simultaneous determination of its value and the restoration of the degraded image. Both proposed iteration adaptive regularization functionals are shown to result in a smoothing functional with a global minimum, so that its iterative optimization does not depend on the initial conditions. The convergence of the algorithm is established and experimental results are shown.     

Regularized constrained total least squares image restoration

In this paper, the problem of restoring an image distorted by a linear space-invariant (LSI) point-spread function (PSF) that is not exactly known is formulated as the solution of a perturbed set of linear equations. The regularized constrained total least-squares (RCTLS) method is used to solve this set of equations. Using the diagonalization properties of the discrete Fourier transform (DFT) for circulant matrices, the RCTLS estimate is computed in the DFT domain. This significantly reduces the computational cost of this approach and makes its implementation possible even for large images. An error analysis of the RCTLS estimate, based on the mean-squared-error (MSE) criterion, is performed to verify its superiority over the constrained total least-squares (CTLS) estimate. Numerical experiments for different errors in the PSF are performed to test the RCTLS estimator. Objective and visual comparisons are presented with the linear minimum mean-squared-error (LMMSE) and the regularized least-squares (RLS) estimator. Our experiments show that the RCTLS estimator reduces significantly ringing artifacts around edges as compared to the two other approaches     

Projection-based spatially adaptive reconstruction ofblock-transform compressed images

At the present time, block-transform coding is probably the most popular approach for image compression. For this approach, the compressed images are decoded using only the transmitted transform data. We formulate image decoding as an image recovery problem. According to this approach, the decoded image is reconstructed using not only the transmitted data but, in addition, the prior knowledge that images before compression do not display between-block discontinuities. A spatially adaptive image recovery algorithm is proposed based on the theory of projections onto convex sets. Apart from the data constraint set, this algorithm uses another new constraint set that enforces between-block smoothness. The novelty of this set is that it captures both the local statistical properties of the image and the human perceptual characteristics. A simplified spatially adaptive recovery algorithm is also proposed, and the analysis of its computational complexity is presented. Numerical experiments are shown that demonstrate that the proposed algorithms work better than both the JPEG deblocking recommendation and our previous projection-based image decoding approach.     

Spatially adaptive iterative algorithm for the restoration of astronomical images

This article develops an iterative spatially adaptive regularized image restoration algorithm. The proposed algorithm is based on the minimization of a weighted smoothing functional. The weighting matrices are defined as functions of the partially restored image at each iteration step. As a result, no prior knowledge about the image and the noise is required, but the weighting matrices as well as the regularization parameter are updated based on the restored image at every step. Conditions for the convexity of the weighted smoothing functional and for the convergence of the iterative algorithm are established for a unique global solution which does not depend on initial conditions. Experimental results are shown with astronomical images which demonstrate the effectiveness of the proposed algorithm.