Temporal error concealment using motion field interpolation

An error concealment algorithm based on motion field interpolation is presented. For each pixel in a damaged block, the algorithm recovers a motion vector using bilinear interpolation of neighbouring motion vectors. This vector is then used to conceal the damaged pixel. Overlapped motion compensation is used to combine this algorithm with a boundary matching error concealment algorithm. Simulation results demonstrate the superior performance of the proposed algorithms     

Hybrid Temporal Error Concealment Methods for Block-Based Compressed Video Transmission

Boundary matching algorithm (BMA) and decoder motion vector estimation (DMVE) are two well-known temporal error concealment methods using the matching-based approach. In these two methods, the motion vector of each missing block is estimated by choosing one among candidate motion vectors which minimizes a sum of absolute differences (SAD) between boundary pixels of the corrupted macroblock. In general, the performance of DMVE is better than that of BMA. However, depending on the location or pattern of the corrupted block, BMA produces higher visual quality than DMVE. In this paper, we propose two types of hybrid error concealment methods; switching method and blending method. The switching method chooses one of two results obtained by BMA and DMVE based on the normalized SAD values. In the blending method, the weighted sum of the results concealed by the aforementioned two methods is utilized to improve the performance of error concealment. In order to reduce blocking artifacts further, the modified overlapped-block motion compensation is adaptively applied to the concealed blocks. Simulation results show that the proposed methods outperform other techniques in terms of subjective visual quality as well as PSNR performance.     

Fuzzy Logic Based Temporal Error Concealment for H.264 Video

In this paper, a new error concealment algorithm is proposed for the H.264 standard. The algorithm consists of two processes. The first process uses a fuzzy logic method to select the size type of lost blocks. The motion vector of a lost block is calculated from the current frame, if the motion vectors of the neighboring blocks surrounding the lost block are discontinuous. Otherwise, the size type of the lost block can be determined from the preceding frame. The second process is an error concealment algorithm via a proposed adapted multiple‐reference‐frames selection for finding the lost motion vector. The adapted multiple‐reference‐frames selection is based on the motion estimation analysis of H.264 coding so that the number of searched frames can be reduced. Therefore the most accurate mode of the lost block can be determined with much less computation time in the selection of the lost motion vector. Experimental results show that the proposed algorithm achieves from 0.5 to 4.52 dB improvement when compared to the method in VM 9.0.     

Recovery of motion vectors for error concealment based on an edge-detection approach

A new approach to error concealment is presented which exploits both spatial and temporal information from the current and previous frames. The technique consists of two stages, motion vector estimation and enhancement of the estimated motion vector. In the first stage the proposed method estimates a replacement for the corrupted motion vector by applying dynamic weights to related motion vectors from the top, bottom, left and right sub-macroblocks. The estimated motion vectors are then enhanced using a new approach based on edge detection in the second stage. The experimental results for several test video sequences are compared with conventional error concealment methods and higher performance is achieved in both objective peak signal-to-noise ratio measurements and subjective visual quality.     

Motion Compensated Three-Dimensional Frequency Selective Extrapolation for improved error concealment in video communication

During transmission of video data over error-prone channels the risk of getting severe image distortions due to transmission errors is ubiquitous. To deal with image distortions at decoder side, error concealment is applied. This article presents Motion Compensated Three-Dimensional Frequency Selective Extrapolation, a novel spatio-temporal error concealment algorithm. The algorithm uses fractional-pel motion estimation and compensation as initial step, being followed by the generation of a model of the distorted signal. The model generation is conducted by an enhanced version of Three-Dimensional Frequency Selective Extrapolation, an existing error concealment algorithm. Compared to this existent algorithm, the proposed one yields an improvement in concealment quality of up to 1.64 dB PSNR. Altogether, the incorporation of motion compensation and the improved model generation extends the already high extrapolation quality of the underlying Frequency Selective Extrapolation, resulting in a gain of more than 3 dB compared to other well-known error concealment algorithms.     

Error resilience of multireference motion compensated prediction in video coding

Standard coded video is vulnerable to transmission errors when transmitted over unreliable channels. The authors propose using multireference motion compensated prediction (MCP) to provide resilience to transmission errors. Both the concealment error and propagation error can be reduced in multireference MCR. First, using the multiple motion vectors associated with each coded block, a concealment scheme of low complexity with small concealment error is proposed. Then the theoretical performance limit of propagation error and coding efficiency in multireference MCP is analysed. Through a numerical solution based on the theoretical performance limit, it is found that a multireference MCP scheme can achieve greater propagation error attenuation than the traditional single-reference MCP scheme. The numerical solution also demonstrates that the performance of the multireference MCP average scheme approaches the theoretical performance limit, with coding efficiency higher than the traditional single-reference MCP scheme. Finally, results of a simulation under the assumed packet loss environment typical of the Internet are given. Simulation results demonstrate that video coding with multireference MCP performs much better than the conventional H.263 coding scheme in a lossy environment.     

Motion field interpolation for temporal error concealment

When transmitted over practical communication channels, compressed video can suffer severe degradation. One approach to combat the effect of channel errors is error concealment. It is an attractive choice because it does not increase the bit rate, it does not require any modifications to the encoder, it does not introduce any delays and it can be applied in almost any application. Conventional temporal concealment techniques estimate one concealment displacement for the whole damaged block and then use translational displacement compensation to conceal the block from a reference frame. The main problem with such techniques is that incorrect estimation of the concealment displacement can lead to poor concealment of the whole or most of the block. Two novel temporal concealment techniques are presented. In the first technique, motion field interpolation is used to estimate one concealment displacement per pel of the damaged block and then each pel is concealed individually. In this case, incorrect estimation of a concealment displacement will only affect the corresponding pel. On a block level this may affect few pels rather than the entire block. In the second technique, multi-hypothesis motion compensation is used to combine the first technique with a boundary matching temporal concealment technique to obtain a more robust performance. Simulation results, within both an isolated error propagation environment and an H.263 codec, show the superior subjective and objective performance of the proposed techniques when compared with conventional temporal concealment techniques     

A Hybrid Frame Concealment Algorithm for H.264/AVC

In packet-based video transmissions, packets loss due to channel errors may result in the loss of the whole video frame. Recently, many error concealment algorithms have been proposed in order to combat channel errors; however, most of the existing algorithms can only deal with the loss of macroblocks and are not able to conceal the whole missing frame. In order to resolve this problem, in this paper, we have proposed a new hybrid motion vector extrapolation (HMVE) algorithm to recover the whole missing frame, and it is able to provide more accurate estimation for the motion vectors of the missing frame than other conventional methods. Simulation results show that it is highly effective and significantly outperforms other existing frame recovery methods.     

A joint encoder–decoder error control framework for stereoscopic video coding

Stereoscopic video coding (SSVC) plays an important role in various 3D video applications. In SSVC, robust stereoscopic video transmission over error-prone networks is still a challenge problem to be solved. In this paper, we propose a joint encoder–decoder error control framework for SSVC, where error-resilient source coding, transmission network conditions, and error concealment scheme are jointly considered to achieve better error robustness performance. The proposed joint encoder–decoder error control framework includes two parts: an error concealment algorithm at the decoder side and a rate–distortion optimized error resilience algorithm at the encoder side. For error concealment at the decoder side, an overlapped block motion and disparity compensation based error concealment scheme is proposed to adaptively utilize inter-view correlations and temporal correlations. For error resilience at the encoder side, first, the inter-view refreshment is proposed for SSVC to suppress error propagations. Then, an end-to-end distortion model for SSVC is derived, which jointly considers the transmission network conditions, inter-view refreshment, and error concealment tools at the decoder side. Finally, based on the derived end-to-end distortion model, the rate–distortion optimized error resilience algorithm is presented to adaptively select inter-view, inter- or intra-coding for SSVC. The experimental results show that the proposed joint encoder–decoder error control framework has superior error robustness performance for stereoscopic video transmission over error-prone networks.     

基于运动预测的Motion JPEG2000误码掩饰新算法

Motion JPEG2000(MJPEG2000)作为新兴的视频压缩标准,鲁棒传输控制是一个重要的应用需求,而误码掩饰技术是提高其鲁棒性的有效途径之一。充分利用压缩码流冗余信息,本文提出了一种基于运动预测的Motion JPEG2000视频传输误码掩饰新算法。首先利用相邻帧及帧内子带间的相关性进行小波域内的运动估计,以低频子带内码块的运动矢量来预测高频子带内丢失码块的运动矢量,再通过运动匹配修复丢失的码块。而低频子带内数据块的丢失可通过整个高频子带图像完成运动估计,得到预测信息对低频子带系数实施误码掩饰。仿真实验结果表明,提出的算法不管在低频还是高频子带进行误码掩饰修复,均能得到较好的图像帧重建效果。     

A variable block size motion estimation algorithm for real-time H.264 video encoding

This paper presents an efficient variable block size motion estimation algorithm for use in real-time H.264 video encoder implementation. In this recursive motion estimation algorithm, results of variable block size modes and motion vectors previously obtained for neighboring macroblocks are used in determining the best mode and motion vectors for encoding the current macroblock. Considering only a limited number of well chosen candidates helps reduce the computational complexity drastically. An additional fine search stage to refine the initially selected motion vector enhances the motion estimator accuracy and SNR performance to a value close to that of full search algorithm. The proposed methods result in over 80% reduction in the encoding time over full search reference implementation and around 55% improvement in the encoding time over the fast motion estimation algorithm (FME) of the reference implementation. The average SNR and compression performance do not show significant difference from the reference implementation. Results based on a number of video sequences are presented to demonstrate the advantage of using the proposed motion estimation technique.     

一种基于快速搜索的视频时域差错隐藏算法

为了克服视频传输中因传输差错引起的视频质量下降,提出一种基于快速搜索的边框匹配时域隐藏算法(CSBM)。该算法针对时域差错隐藏的运动矢量恢复问题,利用边框匹配算法改善被恢复的物体边缘模糊的情况;采用基于中心偏置的快速搜索样式得到最小边界匹配差值的候选运动矢量,减少了解码器差错恢复的计算复杂度。实验结果显示,针对不同性质的序列,该算法与边界匹配算法(SMA)、棱形搜索的边界匹配法(DSSM)等典型差错隐藏方法相比,平均搜索点数可减少12.5~19个点,亮度分量的峰值信噪比(PSNR)能改善0.93~1.55 dB,证明该算法能获得更好的差错隐藏效果,并减少了运算量。     

一种基于多参考帧的时域误码掩盖算法

视频压缩码流对于信道误码十分敏感,可导致重建图像质量严重下降,在接收端就要使用误码掩盖技术对损坏的视频进行掩藏。文中提出一种基于多参考帧的时域误码掩盖算法,首先计算受损宏块的相邻块的运动矢量的均值,然后遍历所有参考帧得到待选的误码掩盖宏块,最后用外边界匹配算法找出替代受损宏块的宏块。研究结果表明,该算法能更精确地恢复错误宏块的运动矢量,从而获得比传统的时域掩盖算法更好的视频质量。     

地面数字电视广播系统中的视频传输容错算法

本文提出了一套针对MPEG-2视频压缩数据传输的自适应容错机制,用于支持地面数字电视广播系统中的可靠电视传输.具体地,本文首先提出了一种基于时域隐藏和块匹配的差错隐藏算法,在保持较好隐藏效果的同时,通过有效地减小搜索窗口而显著降低了算法复杂度.同时,针对独立I帧图像还提出了独特的差错隐藏方法.将这两种差错隐藏算法与低运算复杂度的空域/时域运动检测方法相结合,又衍生出一种自适应空/时域结合差错隐藏算法.仿真结果表明,本文提出的算法即使在很高的数据差错率情况下,仍然可以实现地面数字电视广播系统中可靠而高效的视频传输.     

基于多方向边界匹配的视频误码掩盖算法

视频压缩码流对于信道误码十分敏感,可能导致重建图像质量严重下降.误码掩盖技术利用图像在时间和空间上的相关性,可以有效地降低误码对视频图像的影响.文中提出了一种基于多方向边界匹配的时域误码掩盖算法.该算法能更精确地恢复错误宏块的运动矢量,从而获得比传统的时域掩盖算法更好的视频质量.     

Frame Error Concealment Using Pixel Correlation in Overlapped Motion Compensation Regions

In low bit‐rate video transmission, the payload of a single packet can often contain a whole coded frame due to the high compression ratio in both spatial and temporal domains of most modern video coders. Thus, the loss of a single packet not only causes the loss of a whole frame, but also produces error propagation into subsequent frames. In this paper, we propose a novel whole frame error concealment algorithm which reconstructs the first of the subsequent frames instead of the current lost frame to suppress the effects of error propagation. In the proposed algorithm, we impose a constraint which uses side match distortion (SMD) and overlapped region difference (ORD) to estimate motion vectors between the target reconstructed frame and its reference frame. SMD measures the spatial smoothness connection between a block and its neighboring blocks. ORD is defined as the difference between the correlated pixels which are predicted from one reference pixel. Experimental results show that the proposed algorithm effectively suppresses error propagation and significantly outperforms other conventional techniques in terms of both peak signal‐to‐noise ratio performance and subjective visual quality.     

基于自适应鲁棒性光流的差错掩盖

编码数字视频序列经过受噪声影响的信道传输时,通常会出现图像信息丢失。该文提出一种基于自适应鲁棒性光流的差错掩盖方法,作为解码端的工具解决这样的问题。该文利用光流技术能有效获取物体运动估计的特性,对丢失块进行逐像素点的恢复,既避免了图像模糊,又消除了块效应。在光流的估计中,自适应地调整了目标泛函中的数据保持项与空间连贯项之间的关系,并引入Lorentz函数来构造目标泛函,提高了光流的鲁棒性。仿真结果表明,该文提出的方法无论在主观视觉评价,还是在客观的数值标准下,都能比现有的误差掩盖方法恢复出质量更好的图像。     

Fast Multi-reference Motion Estimation via Enhanced Downhill Simplex Search

Block motion estimation can be regarded as a function minimization problem in a finite-dimensional space. Therefore, fast block motion estimation can be achieved by using an efficient function minimization algorithm instead of a predefined search pattern, such as diamond search. Downhill simplex search is an efficient derivative-free function minimization algorithm. In this paper, we proposed a fast block motion estimation algorithm based on applying the downhill simplex search for function minimization. Several enhanced schemes are proposed to improve the efficiency and accuracy, including a new initialization process, a special rounding scheme, and an early-stop error function evaluation procedure. We also extend the downhill simplex search for the multi-reference frame motion estimation problem. Experimental results show superior performance of the proposed algorithm over some existing fast block matching methods on several benchmarking video sequences.     

An integrated temporal error concealment for H.264/AVC based on spatial evaluation criteria

Owing to error-prone transmission networks, the compressed video bit stream is prone to packet loss in the transmission channel. This loss causes serious distortion and the distortion will propagate to successive frames, especially in highly compressed video coding standard. Therefore, it is very important to efficiently enhance the restored result. In this paper, an integrated temporal error concealment technique for H.264/AVC is proposed. The technique could effectively restore the corrupted data by adaptively integrating error concealment approaches with the adaptive weight-based switching algorithm. The integrated mechanism is based on spatial evaluation criteria, judged by boundary distortion estimation and texture intensity. Experimental results show that the technique could effectively enhance the performance of error concealment.     

一种新的小波可视电话编码算法

针对极低比特率应用提出一种新的结合H.263与SLCCA的混合小波视频编码算法。在提出的算法中,首先,用基于H.263的微调运动估计减小时间冗余,用无遗漏覆盖块运动补偿保证运动补偿误差帧的连续性;第二,对运动补偿误差帧进行小波变换得到全局能量压缩;第三,用SLCCA组织和表示小波变换后的数据,最后,运动向量的水平和垂直分量分别用自适应算法编码,算法在A级测试序列Akiyo和B级测试序列Foreman（QFIF）上测试取得了良好效果。