High‐Performance Spatial and Temporal Error‐Concealment Algorithms for Block‐Based Video Coding Techniques

A compressed video bitstream is sensitive to errors that may severely degrade the reconstructed images even when the bit error rate is small. One approach to combat the impact of such errors is the use of error concealment at the decoder without increasing the bit rate or changing the encoder. For spatial‐error concealment, we propose a method featuring edge continuity and texture preservation as well as low computation to reconstruct more visually acceptable images. Aiming at temporal error concealment, we propose a two‐step algorithm based on block matching principles in which the assumption of smooth and uniform motion for some adjacent blocks is adopted. As simulation results show, the proposed spatial and temporal methods provide better reconstruction quality for damaged images than other methods.     

An error resilient video coding and transmission solution over error-prone channels

With the rapid development of network and multimedia technology, the error control in video coding and video transmission over error-prone channels has become increasingly important. The DCT based predictive coding and VLC based entropy coding greatly increase the coding efficiency, but make the compressed video stream very sensitive to transmission errors. Therefore, this paper proposes a reliable error resilient video coding and video transmission framework. In order to improve the robustness to packet loss errors, an error resilient video coding algorithm named Z-FMO is proposed. Some channel may bring random bit errors, an adaptive error concealment algorithm based on macroblock boundary gradient namely ECMBG is proposed aiming at such problem. As the indispensable part of a video transmission system, we implement an adaptive video transmission control algorithm JCBAF. Experimental results show that the proposed framework performs well both in R-D performance and subjective quality.     

Error concealment techniques for digital TV

Compressed video bitstreams are intended for real-time transmission over communication networks. Most of the video coding standards employ the temporal and spatial prediction structure to reduce the transmitted video data. Therefore, the coded video bitstreams are highly sensitive to information loss and channel errors. Even a single bit error can lead to disastrous quality degradation in both time and space. This quality deterioration is exacerbated when no error resilient coding mechanism is employed to protect coded video data against the error prone environments. Error concealment is a data recovery technique that enables the decoder to conceal effects of transmission errors by predicting the lost or corrupted video data from the previously reconstructed error-free information. Motion vector recovery and motion compensation with the estimated motion vector is a good approach to conceal the corrupted macroblock data. In this paper, we develop various error concealment algorithms based on motion vector recovery, and compare their performances to those of conventional error concealment methods.     

用于多视视频加深度的错误隐藏算法

提出了一种针对多视视频加深度(MVD)的错误隐藏( EC)算法。算法充分利用MVD特有的深度 信息及当前丢失宏块周围正确解码的宏块信息,将丢失宏块分为3类不同属性的宏块。针对3类宏块的特 点,分别提出了基于候选运动矢量修正(CMVR)、基于深度的外边界匹配(DOBMA)以及自适应 权 值的EC(AWEC)等模式。实验表明,本文提出的算法在保证相同的视频主客观质量情况下, 能够快速有效地实现EC。     

Content analysis based smart macroblock rearrangement for error resilience in wireless video transmission

Transmission of compressed video is very sensitive to channel error. The use of resynchronization marks is an efficient method in improving the error resilient performance of a video stream in error-prone environment. In this paper, a smart macroblock rearrangement (SMR) method is proposed to enhance the performance of the resynchronization mark insertion technique for intracoded frames in wireless video transmission. The proposed method makes use of content analysis to rearrange and encode macroblocks between adjacent resynchronization marks. Experiments using the SMR method in both H.263+ and H.264 codecs show that the proposed method outperforms some existing algorithms in both PSNR performance and visual quality.     

一种用于多视视频加深度联合编码的错误隐藏算法

ATM(AVC-based test model)测试模型实现了多视视频加深度(MVD)格式的联合编码,使得数据的压缩效率更高。然而,较高的压缩效率使得码流对传输错误非常敏感,极易产生错误扩散现象。针对ATM测试模型的编码顺序,提出一种用于MVD联合编码的错误隐藏算法。算法充分利用视点内、视点间及纹理视频与深度视频间的相关性,针对每个视点的不同特征提出了适应其视频特性的不同隐藏算法。实验表明,本文提出的算法可以在不增加算法复杂度的情况下,有效提高视频的主客观质量。     

A combined multiple-candidate likelihood decoding and error concealment scheme for compressed video transmission over noisy channels

This paper presents a joint forward error correction (FEC) and error concealment (EC) scheme to enhance the quality of a compressed video signal transmitted over a noisy channel. A multiple candidate likelihood (MCL) channel decoding strategy is used in conjunction with redundancy in the compressed video (syntax validity and spatial discontinuity) to select the best-detected signal.

Simulation results on both objective and subjective performance measures indicate a significant improvement provided by the proposed scheme.     

Error-resilient transcoding for video over wireless channels

We describe a method to maintain quality for video transported over wireless channels. The method is built on three fundamental blocks. First, we use a transcoder that injects spatial and temporal resilience into an encoded bitstream. The amount of resilience is tailored to the content of the video and the prevailing error conditions, as characterized by bit error rate. Second, we derive analytical models that characterize how corruption propagates in a video that is compressed using motion-compensated encoding and subjected to bit errors. Third, we use rate distortion theory to compute the optimal allocation of bit rate among spatial resilience, temporal resilience, and source rate. Furthermore, we use the analytical models to generate the resilience rate distortion functions that are used to compute the optimal resilience. The transcoder then injects this optimal resilience into the bitstream. Simulation results show that using a transcoder to optimally adjust the resilience improves video quality in the presence of errors while maintaining the same input bit rate     

一种基于H.264\AVC的差错隐藏方法

为提高在视频通信中重建图像的质量,针对H.264＼AVC视频编码标准的新特性,提出了一种新的基于H.264＼AVC的差错隐藏方法.受损的帧或者宏块导致的解码错误会在时间上和空间上扩散,该算法充分利用了基于模型的编码和宏块的空间、时间相关性来对受损帧或者宏块进行恢复.仿真结果表明这种方法在不增加比特率的情况下,使峰值信噪比值得到了提高.     

Adaptive shape and texture intra refreshment schemes for improved error resilience in object-based video coding

Video encoders may use several techniques to improve error resilience. In particular, for video encoders that rely on predictive (inter) coding to remove temporal redundancy, intra coding refreshment is especially useful to stop temporal error propagation when errors occur in the transmission or storage of the coded streams, since these errors may cause the decoded quality to decay very rapidly. In the context of object-based video coding, intra coding refreshment can be applied to both the shape and texture data. In this paper, novel shape and texture intra refreshment schemes are proposed which can be used by object-based video encoders, such as MPEG-4 video encoders, independently or combined. These schemes allow to adaptively determine when the shape and texture of the various video objects in a scene should be refreshed in order to maximize the decoded video quality for a certain total bit rate.     

MPEG-2中的错误隐藏技术

由于MPEG-2在编码端引入预测编码,使得MPEG-2视频码流对错误十分敏感。信道传输引起的错误会在时间上扩散,造成图像质量急剧下降。以MPEG-2编码技术为基础,结合实际视频传输的特点,提出了一种基于相邻帧的运动补偿算法和空域多关联线性内插算法,可在解码端实现错误隐藏。仿真实验表明,提出的错误隐藏算法可降低由于误码造成的图像失真和错误扩散,有效地改善了视频图像质量。     

Error concealment for video transmission with dual multiscale Markov random field modeling

A novel error concealment algorithm based on a stochastic modeling approach is proposed as a post-processing tool at the decoder side for recovering the lost information incurred during the transmission of encoded digital video bitstreams. In our proposed scheme, both the spatial and the temporal contextual features in video signals are separately modeled using the multiscale Markov random field (MMRF). The lost information is then estimated using maximum a posteriori (MAP) probabilistic approach based on the spatial and temporal MMRF models; hence, a unified MMRF-MAP framework. To preserve the high frequency information (in particular, the edges) of the damaged video frames through iterative optimization, a new adaptive potential function is also introduced in this paper. Comparing to the existing MRF-based schemes and other traditional concealment algorithms, the proposed dual MMRF (DMMRF) modeling method offers significant improvement on both objective peak signal-to-noise ratio (PSNR) measurement and subjective visual quality of restored video sequence.     

基于多方向边界匹配的视频误码掩盖算法

视频压缩码流对于信道误码十分敏感,可能导致重建图像质量严重下降.误码掩盖技术利用图像在时间和空间上的相关性,可以有效地降低误码对视频图像的影响.文中提出了一种基于多方向边界匹配的时域误码掩盖算法.该算法能更精确地恢复错误宏块的运动矢量,从而获得比传统的时域掩盖算法更好的视频质量.     

An error detection and recovery algorithm for compressed videosignal using source level redundancy

The motion compensation-discrete cosine transform (MC-DCT) coding is an efficient compression technique for a digital video sequence. However, the compressed video signal is vulnerable to transmission errors over noisy channels. In this paper, we propose a robust video transmission algorithm, which protects the compressed video signal by inserting redundant information at the source level. The proposed algorithm encodes every lth frame in the semi-intra frame (S-frame) mode, in which the redundant parity-check DC coefficients (PDCs) are systematically inserted into the compressed bitstream. Then, the decoder is capable of recovering very severe transmission errors, such as loss of an entire frame, in addition to detecting the errors effectively without requesting any information from external devices. The proposed algorithm is implemented based on the H.263 coder, and tested intensively in realistic error prone environment. It is shown that the proposed algorithm provides much better objective and subjective performances than the conventional H.263 coder in the error prone environment.     

基于贝叶斯融合的时空流异常行为检测模型

针对直接利用卷积自编码网络未考虑视频时间信息的问题,该文提出基于贝叶斯融合的时空流异常行为检测模型。空间流模型采用卷积自编码网络对视频单帧进行重构,时间流模型采用卷积长短期记忆(LSTM)编码-解码网络对短期光流序列进行重构。接着,分别计算空间流模型和时间流模型下每帧的重构误差,设计自适应阈值对重构误差图进行二值化,并基于贝叶斯准则对空间流和时间流下的重构误差进行融合,得到融合重构误差图,并在此基础上进行异常行为判断。实验结果表明,该算法在UCSD和Avenue视频库上的检测效果优于现有异常检测算法。     

基于多方向自适应加权的空间错误隐藏算法

为了提高H.264/AVC压缩视频码流在无线信道传输过程中的抗误码性能,提出了一种基于多方向自适应加权的空间错误隐藏算法.该算法首先应用边缘算子检测受损宏块的相邻宏块信息,然后通过多方向自适应加权插值得到受损宏块像素近似值.实验表明,该算法相对传统空间错误隐藏算法,对于不同视频序列图像重构质量有很大的改善,具有较高的应用价值.     

Error-Resilient Performance of Dirac Video Codec Over Packet-Erasure Channel

Video transmission over the wireless or wired network requires error-resilient mechanism since compressed video bitstreams are sensitive to transmission errors because of the use of predictive coding and variable length coding. This paper investigates the performance of a simple and low complexity error-resilient coding scheme which combines source and channel coding to protect compressed bitstream of wavelet-based Dirac video codec in the packet-erasure channel. By partitioning the wavelet transform coefficients of the motion-compensated residual frame into groups and independently processing each group using arithmetic and forward error correction (FEC) coding, Dirac could achieves the robustness to transmission errors by giving the video quality which is gracefully decreasing over a range of packet loss rates up to 30% when compared with conventional FEC only methods. Simulation results also show that the proposed scheme using multiple partitions can achieve up to 10 dB PSNR gain over its existing un-partitioned format. This paper also investigates the error-resilient performance of the proposed scheme in comparison with H.264 over packet-erasure channel.     

视频错误掩盖技术进展

针对2005年到2015年基于H.264/AVC编码标准下的图像错误掩盖技术的发展状况,根据不同类型的编码帧采用的不同掩盖技术,归纳分类了当前已有算法,分析总结了基于传统的错误掩盖技术,即时域掩盖、空域掩盖和时空域结合的掩盖,包括基于场交织的错误掩盖与数字水印技术相结合的错误掩盖,以及基于三维视频的错误掩盖,比较了各算法的优势和局限性,并讨论了其未来发展趋势.