Spatial error concealment with sequence-aligned texture modeling and adaptive directional recovery

A spatial error concealment technique based on the sequence-aligned texture modeling and the adaptive directional recovery is proposed in this work. The sequence alignment technique captures the local variation and the global trend of image textures with surrounding uncorrupted pixels, and provides the best texture model under a given cost function. With the derived texture model, geometric interpolation is used to recover lost pixels adaptively based on pixel locations. There are four candidate modes of pixel sequences to recover lost pixels, and one of them is selected for the concealment purpose. The selection criterion is based on the texture pattern modes of surrounding uncorrupted blocks. The pixel sequences used for error concealment can be obtained from the computation of the decoder or the side information from the encoder. Extensive experimental results are given to demonstrate that our error concealment technique outperforms several benchmark methods in both objective and subjective tests.     

基于最小二乘滤波器的空域视频掩盖算法

针对无线信道传输过程中可能出现的单个或连续宏块丢失现象,提出了一种有效的空-域差错掩盖算法.算法根据丢失像素点中是否含有边界,把丢失的像素点分为平稳点和非平稳点.对于不含有边缘的平稳点,采用线性插值的算法;对于含有边界的非平稳点,建立最小二乘滤波器(LS)模型,通过多方向的扫描方式得出预测值.实验结果表明,算法能够获得优秀的图像质量,在降低了复杂度的同时,适用于视频图像的实时传输.     

一种基于边缘判决的空域差错掩盖算法

提出了一种基于边缘判决的解码视频空域差错掩盖算法。该算法首先对丢失块邻域中的像素进行边缘检测,并对检测出的边缘判断其是否会穿过丢失块。然后使用方向选择过程提取多个候选插值方向,并通过边界像素灰度差决定最终插值方向。实验结果表明本算法有效地提高了错误图像的掩盖效果。     

Efficient decision based algorithm for the removal of high density salt and pepper noise in images

In this paper, an efficient decision based scheme is proposed for the restoration of grayscale and colour images that are heavily corrupted by salt and pepper noise. The processed pixel is examined for 0 or 255; if found true, then it is considered as noisy pixel else not noisy. If found noisy the four neighbours of the noisy pixels are checked for 0 or 255. If all the four neighbours of the corrupted pixel are noisy, the mean of the four neighbours replaces the corrupted pixel. If any of the four neighbours is a non-noisy pixel, calculate the number of corrupted pixels in the current processing window. If the count is less than three then the noisy pixel is replaced by an unsymmetrical trimmed median. If the current window has more than three noisy pixels, then unsymmetrical trimmed mean replaces the corrupted pixels. If all the pixels of the current processing window are noisy then instead of unsymmetrical trimmed mean, global mean of the image is replaced as output. The uncorrupted pixel is left unchanged. The proposed algorithm is tested on various grayscale and colour images and found that it gives excellent PSNR, high IEF and lowest MSE. Also it consumes average time with excellent edge preservation even at higher noise densities. The quality of the results of proposed algorithm is superior when compared to the various state of the art methods.     

Hybrid error concealment method combining exemplar-based image inpainting and spatial interpolation

This paper proposes an efficient error concealment method for the reconstruction of pixels that are lost in video communication. The proposed method is developed by combining exemplar-based image inpainting for patch reconstruction and spatial interpolation for pixel reconstruction using adaptive threshold by local complexity. By exemplar-based image inpainting, regions with regular structures are reconstructed. For complex regions with irregular structures, just one pixel is reconstructed using the proposed spatial interpolation method. The proposed spatial interpolation method performs reconstruction by selecting adaptively directional interpolation or neighbor interpolation based on gradient information. Simulation results show that the proposed hybrid method performs reconstruction with significantly improved subjective quality compared with the previous spatial error concealment and image inpainting methods. The proposed method also gives substantial improvements of PSNR compared with the previous methods.     

Concealment of damaged block transform coded images usingprojections onto convex sets

An algorithm for lost signal restoration in block-based still image and video sequence coding is presented. Problems arising from imperfect transmission of block-coded images result in lost blocks. The resulting image is flawed by the absence of square pixel regions that are notably perceived by human vision, even in real-time video sequences. Error concealment is aimed at masking the effect of missing blocks by use of temporal or spatial interpolation to create a subjectively acceptable approximation to the true error-free image. This paper presents a spatial interpolation algorithm that addresses concealment of lost image blocks using only intra-frame information. It attempts to utilize spatially correlated edge information from a large local neighborhood of surrounding pixels to restore missing blocks. The algorithm is a Gerchberg (1974) type spatial domain/spectral domain constraint-satisfying iterative process, and may be viewed as an alternating projections onto convex sets method.     

Fast DCT-based spatial domain interpolation of blocks in images

An efficient method is presented to restore a block in an image, using a reduced set of border pixels. The basic assumption is that, if the set of pixels consisting of the missing block and its border pixels is transformed by the DCT, the high frequency coefficients obtained can be set to zero. A system of linear equations results, whose solution yields approximations for the values of the lost pixels. In this interpolation scheme, only eight border pixels are used in the computation of any missing pixel. Simulation results show that the approach generally outperforms alternative methods at a smaller computational cost.     

Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation.

The challenge of image interpolation is to preserve spatial details. We propose a soft-decision interpolation technique that estimates missing pixels in groups rather than one at a time. The new technique learns and adapts to varying scene structures using a 2-D piecewise autoregressive model. The model parameters are estimated in a moving window in the input low-resolution image. The pixel structure dictated by the learnt model is enforced by the soft-decision estimation process onto a block of pixels, including both observed and estimated. The result is equivalent to that of a high-order adaptive nonseparable 2-D interpolation filter. This new image interpolation approach preserves spatial coherence of interpolated images better than the existing methods, and it produces the best results so far over a wide range of scenes in both PSNR measure and subjective visual quality. Edges and textures are well preserved, and common interpolation artifacts (blurring, ringing, jaggies, zippering, etc.) are greatly reduced.     

基于最小像素跨度准则的空域错误隐藏方法

提出了一种新的基于最小像素跨度的错误隐藏方法,利用丢失块边界的像素,在无需进行边缘估计的情况下能够有效地恢复出丢失块中可能存在的边缘。本方法主要包括以下三个步骤:1)在最小像素跨度准则下将待恢复像素分为有争议像素和无争议像素;2)对有争议像素和无争议像素使用不同方法分别恢复;3)对整个丢失块做滤波处理以消除上述过程中的噪点。试验表明本方法有效地保留边缘,并在 PSNR 上优于其它传统方法 1～2dB。     

Temporal error concealment using motion field interpolation

An error concealment algorithm based on motion field interpolation is presented. For each pixel in a damaged block, the algorithm recovers a motion vector using bilinear interpolation of neighbouring motion vectors. This vector is then used to conceal the damaged pixel. Overlapped motion compensation is used to combine this algorithm with a boundary matching error concealment algorithm. Simulation results demonstrate the superior performance of the proposed algorithms     

An effective error concealment scheme for heavily corrupted H.264/AVC videos based on Kalman filtering

Error concealment at the decoder side is an economical approach to ensuring an acceptable and stable video quality in case of packet erasure or loss, and thus, it has attracted considerable research and application interest. Relevant techniques usually employ the spatial or temporal correlation to recover the motion vectors (MVs) of the missing blocks, and interpolation, extrapolation, or boundary-matching schemes are usually effective. However, for heavily corrupted sequences, e.g., with block loss rate beyond 50 %, most methods might perform less satisfactorily. Inspired by the tracking efficiency of Kalman filter (KF), in the present work, we adopted it to predict the missing MVs, and the unpredicted ones (minority) were restored complementarily using a modified bilinear motion field interpolation (MFI) method. Since the KF prediction is independent of the loss rate, the present framework proves to be robust for heavily corrupted videos. Experimental results on typical sequences reveal that the proposed algorithm outperforms the boundary-matching algorithm embedded in the H.264/AVC reference code, the MFI and the MV extrapolation techniques in the literature.     

Hybrid Temporal Error Concealment Methods for Block-Based Compressed Video Transmission

Boundary matching algorithm (BMA) and decoder motion vector estimation (DMVE) are two well-known temporal error concealment methods using the matching-based approach. In these two methods, the motion vector of each missing block is estimated by choosing one among candidate motion vectors which minimizes a sum of absolute differences (SAD) between boundary pixels of the corrupted macroblock. In general, the performance of DMVE is better than that of BMA. However, depending on the location or pattern of the corrupted block, BMA produces higher visual quality than DMVE. In this paper, we propose two types of hybrid error concealment methods; switching method and blending method. The switching method chooses one of two results obtained by BMA and DMVE based on the normalized SAD values. In the blending method, the weighted sum of the results concealed by the aforementioned two methods is utilized to improve the performance of error concealment. In order to reduce blocking artifacts further, the modified overlapped-block motion compensation is adaptively applied to the concealed blocks. Simulation results show that the proposed methods outperform other techniques in terms of subjective visual quality as well as PSNR performance.     

Improved interview video error concealment on whole frame packet loss

An improved DIBR-based (Depth image based rendering) whole frame error concealment method for multiview video with depth is designed. An optimal reference view selection is first proposed. The paper further includes three modified parts for the DIBRed pixels. First, the missing 1-to-1 pixels are concealed by the pixels from another view. The light differences between views are taken care of by the information of the motion vector of the projected coordination and a reverse DIBR procedure. Second, the generation of the many-to-1 pixels is improved via their depth information. Third, the hole pixels are found using the estimated motion vectors derived efficiently from a weighted function of the neighboring available motion vectors and their distance to the target hole pixel. The experimental results show that, compared to the state-of-the-art method, the combined system of the four proposed methods is superior and improves the performance by 5.53 dB at maximum.     

Content-based adaptive spatio-temporal methods for MPEG repair

Block loss and propagation error due to cell loss or missing packet information during the transmission over lossy networks can cause severe degradation of block and predictive-based video coding. Herein, new fast spatial and temporal methods are presented for block loss recovery. In the spatial algorithm, missing block recovery and edge extention are performed by pixel replacement based on range constraints imposed by surrounding neighborhood edge information and structure. In the temporal algorithm, an adaptive temporal correlation method is proposed for motion vector (MV) recovery. Parameters for the temporal correlation measurement are adaptively changed in accordance to surrounding edge information of a missing macroblock (MB). The temporal technique utilizes pixels in the reference frame as well as surrounding pixels of the lost block. Spatial motion compensation is applied after MV recovery when the reference frame does not have sufficient information for lost MB restoration. Simulations demonstrate that the proposed algorithms recover image information reliably using both spatial and temporal restoration. We compare the proposed algorithm with other procedures with consistently favorable results.     

Mode selection algorithms for enhanced spatiotemporal error concealment

Mode selection algorithms for spatiotemporal concealment of missing macroblocks (MBs) in a corrupted video stream are presented. The proposed algorithms employ the motion compensated temporal activity in the neighbourhood of each missing MB for switching between concealment methods. Results show gains of up to 4.6 dB for a 10% packet error rate relative to other mode selection methods     

The successively temporal error concealment algorithm using error-adaptive block matching principle

Generally, the temporal error concealment (TEC) adopts the blocks around the corrupted block (CB) as the search pattern to find the best-match block in previous frame. Once the CB is recovered, it is referred to as the recovered block (RB). Although RB can be the search pattern to find the best-match block of another CB, RB is not the same as its original block (OB). The error between the RB and its OB limits the performance of TEC. The successively temporal error concealment (STEC) algorithm is proposed to alleviate this error. The STEC procedure consists of tier-1 and tier-2. The tier-1 divides a corrupted macroblock into four corrupted 8 × 8 blocks and generates a recovering order for them. The corrupted 8 × 8 block with the first place of recovering order is recovered in tier-1, and remaining 8 × 8 CBs are recovered in tier-2 along the recovering order. In tier-2, the error-adaptive block matching principle (EA-BMP) is proposed for the RB as the search pattern to recover remaining corrupted 8 × 8 blocks. The proposed STEC outperforms sophisticated TEC algorithms on average PSNR by 0.3 dB on the packet error rate of 20% at least.     

Adaptive homogeneity-directed demosaicing algorithm.

A cost-effective digital camera uses a single-image sensor, applying alternating patterns of red, green, and blue color filters to each pixel location. A way to reconstruct a full three-color representation of color images by estimating the missing pixel components in each color plane is called a demosaicing algorithm. This paper presents three inherent problems often associated with demosaicing algorithms that incorporate two-dimensional (2-D) directional interpolation: misguidance color artifacts, interpolation color artifacts, and aliasing. The level of misguidance color artifacts present in two images can be compared using metric neighborhood modeling. The proposed demosaicing algorithm estimates missing pixels by interpolating in the direction with fewer color artifacts. The aliasing problem is addressed by applying filterbank techniques to 2-D directional interpolation. The interpolation artifacts are reduced using a nonlinear iterative procedure. Experimental results using digital images confirm the effectiveness of this approach.     

A new fuzzy-based decision algorithm for high-density impulse noise removal

This paper proposes a new efficient fuzzy-based decision algorithm (FBDA) for the restoration of images that are corrupted with high density of impulse noises. FBDA is a fuzzy-based switching median filter in which the filtering is applied only to corrupted pixels in the image while the uncorrupted pixels are left unchanged. The proposed algorithm computes the difference measure for each pixel based on the central pixel (corrupted pixel) in a selected window and then calculates the membership value for each pixel based on the highest difference. The algorithm then eliminates those pixels from the window with very high and very low membership values, which might represent the impulse noises. Median filter is then applied to the remaining pixels in the window to get the restored value for the current pixel position. The proposed algorithm produces excellent results compared to conventional method such as standard median filter (SMF) as well as some advanced techniques such as adaptive median filters (AMF), efficient decision-based algorithm (EDBA), improved efficient decision-based algorithm (IDBA) and boundary discriminative noise detection (BDND) switching median filter. The efficiency of the proposed algorithm is evaluated using different standard images. From experimental analysis, it has been found that FBDA produces better results in terms of both quantitative measures such as PSNR, SSIM, IEF and qualitative measures such as Image Quality Index (IQI).     

Similar neighbor criterion for impulse noise removal in images

In this paper, a new method is proposed for removing and restoring random-valued impulse noise in images. This approach is based on a similar neighbor criterion, in which any pixel to be considered as an original pixel it should have sufficient numbers of similar neighboring pixels in a set of filtering windows. Compared with other well known methods in the literature, this technique achieves superior performance in restoring heavily corrupted noisy images. Furthermore, it has low computational complexity, and equally effective in restoring corrupted color and gray-level images.     

High performance de-interlacing algorithm for digital television displays

In this paper, a high performance de-interlacing algorithm is proposed to reduce the artifact effects produced by interlaced scanning TV systems. In the proposed algorithm, a moving-stationary detector is presented to determine where the missing pixels belong to moving or stationary region. If the missing pixel belongs to the stationary region, the temporal-wise interpolation is adopted; otherwise, the spatial-temporal-wise interpolation is applied. In addition, the proposed algorithm has simple operations chiefly involves addition and subtraction. This simple computation conducts the proposed algorithm to real-time applications efficiently. Experimental results show that the proposed algorithm can improve the subjective and objective qualities compared with other de-interlacing algorithms.