Error concealment algorithms for robust decoding of MPEG compressed video

This paper provides some research results on the topic of error resilience for robust decoding of MPEG (Moving Picture Experts Group) compressed video. It introduces and characterizes the performance of a general class of error concealment algorithms. Such receiver-based error concealment techniques are essential for many practical video transmission scenarios such as terrestrial HDTV broadcasting, packet network based teleconferencing/multimedia, and digital SDTV/HDTV delivery via ATM (asynchronous transfer mode). Error concealment is intended to ameliorate the impact of channel impairments (i.e., bit-errors in noisy channels, or cell-loss in ATM networks) by utilizing available picture redundancy to provide a subjectively acceptable rendition of affected picture regions. The concealment process must be supported by an appropriate transport format which helps to identify image pixel regions which correspond to lost or damaged data. Once the image regions (i.e., macroblocks, slices, etc.) to be concealed are identified, a combination of spatial and temporal replacement techniques may be applied to fill in lost picture elements. A specific class of spatio-temporal error concealment algorithms for MPEG video is described and alternative realizations are compared via detailed end-to-end simulations for both one- or two-tier transmission media. Several algorithm enhancements based on directional interpolation, ‘I-picture motion vectors’, and use of MPEG-2 ‘scalability’ features are also presented. In each case, achievable performance improvements are estimated via simulation. Overall, these results demonstrate that the proposed class of error concealment algorithms provides significant robustness for MPEG video delivery in the presence of channel impairments, permitting useful operation at ATM cell-loss rates in the region of 10⁻⁴ to 10⁻³ and 10⁻² to 10⁻¹ for one- and two-tier transmission scenarios, respectively.     

Two-dimensional reversible data hiding-based approach for intra-frame error concealment in H.264/AVC

Highly compressed video bit-stream is extremely sensitive to transmission error. A novel two-dimensional reversible data hiding-based approach for intra-frame error concealment is proposed, which aims at improving video quality at decoder when video bit-stream incur transmission errors. The scheme involves embedding the motion vector (MV) of a macroblock (MB) into other MB within the same intra-frame, and extracting the embedded MV from the received video frame for reconstruction of the corrupted MB. Based on the distribution of the motion vector data and the characteristic of histogram shifting, a specific two-dimensional reversible data hiding mechanism is designed. Consequently, the distortion of the marked video can be controlled at a low level. Experimental results show that the damaged macroblocks can be recovered with a higher quality using the reversible data hiding methodology, as compared to the non-reversible data hiding method. Furthermore, experimental results demonstrate that the proposed algorithm outperforms some state-of-the-art reversible data hiding error concealment schemes in improving the perceptual quality.     

一种移动视频通信中的错误隐藏方法研究

无线视频移动信道是高误码率，具有突发错误的信道。高效压缩的视频信号对传输过程中的错误非常敏感，容易造成宏块连续出错，不利于错误隐藏的有效实施。本文提出一种基于宏块的交错扫描方法，使得相邻宏块不同时发生错误。分析和实验表明．这一方法对出错视频图像的恢复具有明显的有效性。     

Perceptually driven video error protection using a distributed source coding approach

In video communication systems, the video signals are typically compressed and sent to the decoder through an error-prone transmission channel that may corrupt the compressed signal, causing the degradation of the final decoded video quality. In this context, it is possible to enhance the error resilience of typical predictive video coding schemes using as inspiration principles and tools from an alternative video coding approach, the so-called Distributed Video Coding (DVC), based on the Distributed Source Coding (DSC) theory. Further improvements in the decoded video quality after error-prone transmission may also be obtained by considering the perceptual relevance of the video content, as distortions occurring in different regions of a picture have a different impact on the user's final experience. In this context, this paper proposes a Perceptually Driven Error Protection (PDEP) video coding solution that enhances the error resilience of a state-of-the-art H.264/AVC predictive video codec using DSC principles and perceptual considerations. To increase the H.264/AVC error resilience performance, the main technical novelties brought by the proposed video coding solution are: (i) design of an improved compressed domain perceptual classification mechanism; (ii) design of an improved transcoding tool for the DSC-based protection mechanism; and (iii) integration of a perceptual classification mechanism in an H.264/AVC compliant codec with a DSC-based error protection mechanism. The performance results obtained show that the proposed PDEP video codec provides a better performing alternative to traditional error protection video coding schemes, notably Forward Error Correction (FEC)-based schemes.     

3D视频的错误隐藏快速算法

具有独特编码结构的3D视频在不可靠信道上传输时遭遇丢包后极易造成错误传播,因此研究基于3D视频的错误隐藏技术十分必要。而且,压缩后传输的3D视频数据量仍然较大,这对解码器的解码速度提出了很高的要求。如何在解码端加速进行错误隐藏是研究重点,通过快速定位静止宏块的方法加速错误隐藏速度。实验结果表明该算法在保证不降低视频的主、客观质量的同时,能有效地降低错误隐藏算法的复杂度。     

High‐Performance Spatial and Temporal Error‐Concealment Algorithms for Block‐Based Video Coding Techniques

A compressed video bitstream is sensitive to errors that may severely degrade the reconstructed images even when the bit error rate is small. One approach to combat the impact of such errors is the use of error concealment at the decoder without increasing the bit rate or changing the encoder. For spatial‐error concealment, we propose a method featuring edge continuity and texture preservation as well as low computation to reconstruct more visually acceptable images. Aiming at temporal error concealment, we propose a two‐step algorithm based on block matching principles in which the assumption of smooth and uniform motion for some adjacent blocks is adopted. As simulation results show, the proposed spatial and temporal methods provide better reconstruction quality for damaged images than other methods.     

Error concealment aware rate shaping for wireless video transport

Streaming of video, which is both source- and channel-coded, over wireless networks faces the challenge of time-varying packet loss rate and fluctuating bandwidth. Rate shaping (RS) has been proposed to reduce the bit-rate of a precoded video bitstream to adapt to the real-time bandwidth variation. In our earlier work, rate shaping was extended to consider not only the bandwidth but also the packet loss rate variations. Rate-distortion optimized rate adaptation is performed on the precoded video that is a scalable coded bitstream protected by forward error correction codes. In this paper, we propose an RS scheme that further takes into account the error concealment (EC) method used at the receiver. We refer to this scheme as EC aware RS (ECARS). When performing ECARS, first ECARS needs to know the benefit/gain of sending each part of the precoded video, as opposed to not sending it but reconstructing it by EC. Then given a certain packet loss probability, the expected gain can be derived and be included in the rate-distortion optimization problem formulation. Finally, ECARS performs rate-distortion optimization to adapt the rate of the precoded video. A two-stage rate-distortion optimization approach is proposed to solve the ECARS rate-distortion optimization problem. In addition to ECARS, the precoding process can be EC aware to prioritize the precoded video based on the gain. We present an example EC aware precoding process by means of macroblock prioritization. Experiment results of ECARS together with EC aware precoding are shown to have excellent performance.     

Error prevention and concealment for scalable video coding with dual-priority transmission

In this work, we present an efficient error resilient system against ATM cell loss using a hybrid error concealment and error propagation prevention (ECP) technique with dual-priority transmission scheme (DPTS). DPTS performs traffic policing to form dual-priority cells in ATM connections and manages to make most cell losses occur in a low priority layer. However, cell loss may still occur in the high priority layer if the bandwidth is not reserved enough for the usually variable bitrate video traffic. Therefore, the ECP technique can still be utilized to reduce the error damage and limit the impact of cell loss to the erroneous slices. Simulation results of two-layer MPEG-2 coding over DPTS in ATM networks demonstrate that ECP with feedback over DPTS can effectively isolate errors and reduce the damage to yield a satisfactory performance, even when the cell-loss rate is as high as 8%.     

IP视频通信中的错误掩盖技术

当视频压缩数据在QoS(Quality of Service)不能保障的IP网络(如INTERNET)上传输时,必须考虑信道所引入错误的影响,错误掩盖是消除错误的重要方法.本文建议了一种能够保持图像边缘的空域错误掩盖算法,该算法适用于基于块的视频压缩编码标准,如MPEG 2、H.263等.该算法根据受损宏块邻近区域的边缘信息自适应地确定用于内插的像素的权值,实验证明该算法能够很好地保持边缘的连续性,达到了较好的图像掩盖效果.     

基于H.264 SVC无线视频传输的自适应差错保护方法

提出了一种针对H.264可分级编码(H.264 SVC)的自适应前向纠错编码保护方案.通过比较不同的纠错方案,提出了划分丢包率区间的概念,并根据不同区间的丢包率自适应地选择最佳的纠错方案.仿真结果表明,与单一保护方法相比,所提自适应方法能够取得更好的保护效果,更适于在无线信道中进行视频传输.     

基于人脸特征的自适应空域差错掩盖算法

针对在视频通信中,高压缩视频码流对通信信道中由于信道带宽、环境等影响出现的丢包和误码极为敏感,本文提出一种在解码端基于人脸检测的自适应空域差错掩盖算法,根据正确接收帧的人脸位置及人脸运动的趋势预测受损帧人脸的位置,对于受损帧人脸内五官区域的宏块采用水平双线性插值,对于背景区域宏块采用自适应多方向插值算法。实验结果表明与现有算法相比主客观质量都有所提高。     

视频通信中的误码掩盖技术初探

在数字视频通信中,由于信道不理想等原因容易产生误码,而且误码在空间和时间上的扩散会导致重建图像质量不同程度的下降。最近发展起来的误码掩盖技术就是用来解决这类问题、改进重建视频质量的重要方法之一。在简要介绍误码掩盖的基本原理和相关技术的基础上,着重分析了新近采用的几种误码掩盖方法的应用范围和性能,并指出了它们的发展前景。．     

基于多方向自适应加权的空间错误隐藏算法

为了提高H.264/AVC压缩视频码流在无线信道传输过程中的抗误码性能,提出了一种基于多方向自适应加权的空间错误隐藏算法.该算法首先应用边缘算子检测受损宏块的相邻宏块信息,然后通过多方向自适应加权插值得到受损宏块像素近似值.实验表明,该算法相对传统空间错误隐藏算法,对于不同视频序列图像重构质量有很大的改善,具有较高的应用价值.     

Error Concealment Based on Semantic Prioritization with Hardware‐Based Face Tracking

With video compression standards such as MPEG‐4, a transmission error happens in a video‐packet basis, rather than in a macroblock basis. In this context, we propose a semantic error prioritization method that determines the size of a video packet based on the importance of its contents. A video packet length is made to be short for an important area such as a facial area in order to reduce the possibility of error accumulation. To facilitate the semantic error prioritization, an efficient hardware algorithm for face tracking is proposed. The increase of hardware complexity is minimal because a motion estimation engine is efficiently re‐used for face tracking. Experimental results demonstrate that the facial area is well protected with the proposed scheme.     

Error-Resilient Performance of Dirac Video Codec Over Packet-Erasure Channel

Video transmission over the wireless or wired network requires error-resilient mechanism since compressed video bitstreams are sensitive to transmission errors because of the use of predictive coding and variable length coding. This paper investigates the performance of a simple and low complexity error-resilient coding scheme which combines source and channel coding to protect compressed bitstream of wavelet-based Dirac video codec in the packet-erasure channel. By partitioning the wavelet transform coefficients of the motion-compensated residual frame into groups and independently processing each group using arithmetic and forward error correction (FEC) coding, Dirac could achieves the robustness to transmission errors by giving the video quality which is gracefully decreasing over a range of packet loss rates up to 30% when compared with conventional FEC only methods. Simulation results also show that the proposed scheme using multiple partitions can achieve up to 10 dB PSNR gain over its existing un-partitioned format. This paper also investigates the error-resilient performance of the proposed scheme in comparison with H.264 over packet-erasure channel.     

Robust Digital Terrestrial TV Broadcasting System with Error-Resilient Schemes

In this paper, error-resilient schemes are proposed to support robust video transmission for digital terrestrial TV broadcasting (DTTB). In particular, a temporal error concealment incorporated with a low-complexity block-matching is developed, achieving an effective reception of predictive pictures in harsh terrestrial environment. Special algorithms are also designed for isolated I-pictures. Moreover, combined with an intra/inter case prediction, an adaptive error concealment scheme is further contrived to fit for different error conditions. Extensive simulations have been conducted under various DTTB channel conditions, even with a very high packet error rate, to verify the effectiveness of the proposed schemes.     

无线视频传输中FEC与WZ联合抗误码算法研究

无线视频编码的低抗误性、无线信道带宽的有限性、无线视频业务的高需求决定了高鲁棒性的抗误算法必将成为无线视频通信的核心问题之一.基于FEC与WZ (Wyner-Ziv)两抗误工具的在不同误码率下抗误互补特性,提出了FEC与WZ联合抗误码方案.实验表明:该方案有效地综合了两者优点,提升了平均PSNR.     

H.263编码视频流的混合错误掩盖

当H．263编码视频流在因特网上传输时，易受错误的影响，错误不但会影响当前帧还会继续扩散到以后的解码帧，从而导致图像质量的严重恶化，目前消除错误影响的常用算法是空域掩盖和时域掩盖算法，但是单纯地使用空域算法会造成图像的钝化，而时域算法则无法处理大运动的图像区域，因此，建议了一种时域和空域混合掩盖算法，同时使用两类算法对发生错误的图像帧进行掩盖。模拟结果显示该算法能够达到可以接受的图像质量，适应于视频会议、远程教育等应用的要求。