A joint encoder–decoder error control framework for stereoscopic video coding

Stereoscopic video coding (SSVC) plays an important role in various 3D video applications. In SSVC, robust stereoscopic video transmission over error-prone networks is still a challenge problem to be solved. In this paper, we propose a joint encoder–decoder error control framework for SSVC, where error-resilient source coding, transmission network conditions, and error concealment scheme are jointly considered to achieve better error robustness performance. The proposed joint encoder–decoder error control framework includes two parts: an error concealment algorithm at the decoder side and a rate–distortion optimized error resilience algorithm at the encoder side. For error concealment at the decoder side, an overlapped block motion and disparity compensation based error concealment scheme is proposed to adaptively utilize inter-view correlations and temporal correlations. For error resilience at the encoder side, first, the inter-view refreshment is proposed for SSVC to suppress error propagations. Then, an end-to-end distortion model for SSVC is derived, which jointly considers the transmission network conditions, inter-view refreshment, and error concealment tools at the decoder side. Finally, based on the derived end-to-end distortion model, the rate–distortion optimized error resilience algorithm is presented to adaptively select inter-view, inter- or intra-coding for SSVC. The experimental results show that the proposed joint encoder–decoder error control framework has superior error robustness performance for stereoscopic video transmission over error-prone networks.     

Feedback-based error control for mobile video transmission

We review feedback-based low bit-rate video coding techniques for robust transmission in mobile multimedia networks. For error control on the source coding level, each decoder has to make provisions for error detection, resynchronization, and error concealment, and we review techniques suitable for that purpose. Further, techniques are discussed for intelligent processing of acknowledgment information by the coding control to adapt the source coder to the channel. We review and compare error tracking, error confinement, and reference picture selection techniques for channel-adaptive source coding. For comparison of these techniques, a system for transmitting low bit-rate video over a wireless channel is presented and the performance is evaluated for a range of transmission conditions. We also show how feedback-based source coding can be employed in conjunction with precompressed video stored on a media server. The techniques discussed are applicable to a wide variety of interframe video schemes, including various video coding standards. Several of the techniques have been incorporated into the H.263 video compression standard, and this standard is used as an example throughout     

H.264视频误码掩盖综述

在无线网络和因特网上传输视频，误码掩盖的问题越来越重要。回顾了文献中介绍的误码掩盖的方法。误码掩盖后处理利用图像和视频信号的相关特性，在解码端对丢失区域进行掩盖，并总结了空域、时域，以及时空结合的误码掩盖方法。     

3D视频的错误隐藏快速算法

具有独特编码结构的3D视频在不可靠信道上传输时遭遇丢包后极易造成错误传播,因此研究基于3D视频的错误隐藏技术十分必要。而且,压缩后传输的3D视频数据量仍然较大,这对解码器的解码速度提出了很高的要求。如何在解码端加速进行错误隐藏是研究重点,通过快速定位静止宏块的方法加速错误隐藏速度。实验结果表明该算法在保证不降低视频的主、客观质量的同时,能有效地降低错误隐藏算法的复杂度。     

Error resilient video coding techniques

We review error resilience techniques for real-time video transport over unreliable networks. Topics covered include an introduction to today's protocol and network environments and their characteristics, encoder error resilience tools, decoder error concealment techniques, as well as techniques that require cooperation between encoder, decoder, and the network. We provide a review of general principles of these techniques as well as specific implementations adopted by the H.263 and MPEG-4 video coding standards. The majority of the article is devoted to the techniques developed for block-based hybrid coders using motion-compensated prediction and transform coding. A separate section covers error resilience techniques for shape coding in MPEG-4     

基于Internet的实时视频流的应用层QoS控制策略

分析了Internet上实时视频传输的特点，提出了基于Internet的实时视频流的应用层QoS控制策略，主要包括拥塞控制策略和差错控制策略以及相应的控制技术。在拥塞控制中，讨论速率控制和速率整形，速率控制主要是根据网络运行状态预测当前可用的带宽，并根据预测值调整视频速率，达到与可用带宽匹配；速率整形则是迫使发送端以码率控制算法规定的码率发送视频流。在差错控制中，则讨论了编码器差错复原、解码器错误隐藏和编码器／解码器交互的差错控制等控制策略。这些控制技术应用于终端系统并不需要路由器和网络的QoS支持，可以最大限度地提高视频质量。     

Optimal mode selection and synchronization for robust videocommunications over error-prone networks

We describe an effective method for increasing error resilience of video transmission over bit error prone networks. Rate-distortion optimized mode selection and synchronization marker insertion algorithms are introduced. The resulting video communication system takes into account the channel condition and the error concealment method used by the decoder, to optimize video coding mode selection and placement of synchronization markers in the compressed bit stream. The effects of mismatch between the parameters used by the encoder and the parameters associated with the actual channel condition and the decoder error concealment method are evaluated. Results for the binary symmetric channel and wideband code division multiple access mobile network models are presented in order to illustrate the advantages of the proposed method     

Content-Aware Resource Allocation and Packet Scheduling for Video Transmission over Wireless Networks

A cross-layer packet scheduling scheme that streams pre-encoded video over wireless downlink packet access networks to multiple users is presented. The scheme can be used with the emerging wireless standards such as HSDPA and IEEE 802.16. A gradient based scheduling scheme is used in which user data rates are dynamically adjusted based on channel quality as well as the gradients of a utility function. The user utilities are designed as a function of the distortion of the received video. This enables distortion-aware packet scheduling both within and across multiple users. The utility takes into account decoder error concealment, an important component in deciding the received quality of the video. We consider both simple and complex error concealment techniques. Simulation results show that the gradient based scheduling framework combined with the content-aware utility functions provides a viable method for downlink packet scheduling as it can significantly outperform current content-independent techniques. Further tests determine the sensitivity of the system to the initial video encoding schemes, as well as to non-real-time packet ordering techniques.     

Error concealment techniques for digital TV

Compressed video bitstreams are intended for real-time transmission over communication networks. Most of the video coding standards employ the temporal and spatial prediction structure to reduce the transmitted video data. Therefore, the coded video bitstreams are highly sensitive to information loss and channel errors. Even a single bit error can lead to disastrous quality degradation in both time and space. This quality deterioration is exacerbated when no error resilient coding mechanism is employed to protect coded video data against the error prone environments. Error concealment is a data recovery technique that enables the decoder to conceal effects of transmission errors by predicting the lost or corrupted video data from the previously reconstructed error-free information. Motion vector recovery and motion compensation with the estimated motion vector is a good approach to conceal the corrupted macroblock data. In this paper, we develop various error concealment algorithms based on motion vector recovery, and compare their performances to those of conventional error concealment methods.     

Combined and iterative form of spatial and temporal error concealment for video signals

Video transmission over unreliable channels may suffer from the data loss or corruption. To facilitate the transmission, video content is compressed to save the bandwidth. The compression algorithms remove the redundancies in the video signal, meanwhile increasing the dependencies among symbols in the compressed bit-stream. Thus, when errors occur, they may propagate in both space and time. Among various error control techniques, error concealment (EC) is an effective method that is performed at the decoder to mitigate the influence of errors on the quality of reconstructed images. In this paper, a joint spatial and temporal EC algorithm (JSTEC) is presented. First, several existing temporal EC algorithms are combined in an appropriate order. Then, the spatial correlation in the video is exploited in an iterative form to improve the performance of the temporal EC. Experimental results show that JSTEC performs better both in peak signal-to-noise ratio and subjective quality of images than the existing algorithms.     

Analysis of video transmission over lossy channels

A theoretical analysis of the overall mean squared error (MSE) in hybrid video coding is presented for the case of error prone transmission. Our model covers the complete transmission system including the rate-distortion performance of the video encoder, forward error correction, interleaving, and the effect of error concealment and interframe error propagation at the video decoder. The channel model used is a 2-state Markov model describing burst errors on the symbol level. Reed-Solomon codes are used for forward error correction. Extensive simulation results using an H.263 video codec are provided for verification. Using the model, the optimal tradeoff between INTRA and INTER coding as well as the optimal channel code rate can be determined for given channel parameters by minimizing the expected MSE at the decoder. The main focus of this paper is to show the accuracy of the derived analytical model and its applicability to the analysis and optimization of an entire video transmission system     

一种基于率失真优化的关键参考帧选择算法

为抑制压缩视频流的传输错误扩散,提出了一种新的基于关键帧的无反馈参考帧选择算法,通过采用一种适用各种掩盖策略的端到端失真度估计方法,评价出选择不同参考帧情况下的解码器失真度期望,在率失真框架内选择最合适的参考帧,使解码端的失真度达到最小.基于H. 263+ TMN8测试模型和IP包丢失的模拟实验表明,提出的率失真优化关键参考帧算法比传统的预测编码方法,PSNR值大约提高了1~3dB.当数据丢包率严重时,优势尤为明显.     

Proposed dynamic error control techniques for QoS improvement of wireless 3D video transmission

Error control techniques like error resilience (ER) and error concealment (EC) are efficient techniques to ameliorate the lost macroblocks (MBs) in the 3D video (3DV) communication system. In this paper, we propose efficient and adaptive hybrid ER‐EC algorithms for 3DV transmission over error‐prone wireless channels. At the encoder, adaptive preprocessing ER mechanisms are proposed through using the context adaptive variable length coding entropy, slice structured coding modes, and explicit flexible macroblock ordering mapping. They are used to assist the suggested EC techniques at the decoder to accurately reconstruct the erroneous MBs and frames. At the decoder, an efficient postprocessing EC technique with multiproposition methods is proposed to dynamically select the convenient EC hypothesis method based on the size of the lost MBs, the faulty view, and the frame type. It conceals the received erroneous MBs of intra‐encoded and inter‐encoded frames of the transmitted 3DV by exploiting the temporal, spatial, and inter‐view correlations among frames and views. To further improve the decoded 3DV quality, a weighted overlapping block motion and disparity compensation technique is used to reinforce the performance of the suggested ER‐EC techniques. Experimental results on various 3DV streams prove that the suggested techniques have considerably acceptable subjective and objective 3DV performance. They achieve an improved average peak signal‐to‐noise ratio gain by almost 2.85 dB compared to the conventional error control algorithms at a packet loss rate = 40%.     

Two-dimensional reversible data hiding-based approach for intra-frame error concealment in H.264/AVC

Highly compressed video bit-stream is extremely sensitive to transmission error. A novel two-dimensional reversible data hiding-based approach for intra-frame error concealment is proposed, which aims at improving video quality at decoder when video bit-stream incur transmission errors. The scheme involves embedding the motion vector (MV) of a macroblock (MB) into other MB within the same intra-frame, and extracting the embedded MV from the received video frame for reconstruction of the corrupted MB. Based on the distribution of the motion vector data and the characteristic of histogram shifting, a specific two-dimensional reversible data hiding mechanism is designed. Consequently, the distortion of the marked video can be controlled at a low level. Experimental results show that the damaged macroblocks can be recovered with a higher quality using the reversible data hiding methodology, as compared to the non-reversible data hiding method. Furthermore, experimental results demonstrate that the proposed algorithm outperforms some state-of-the-art reversible data hiding error concealment schemes in improving the perceptual quality.     

一种用于多视视频加深度联合编码的错误隐藏算法

ATM(AVC-based test model)测试模型实现了多视视频加深度(MVD)格式的联合编码,使得数据的压缩效率更高。然而,较高的压缩效率使得码流对传输错误非常敏感,极易产生错误扩散现象。针对ATM测试模型的编码顺序,提出一种用于MVD联合编码的错误隐藏算法。算法充分利用视点内、视点间及纹理视频与深度视频间的相关性,针对每个视点的不同特征提出了适应其视频特性的不同隐藏算法。实验表明,本文提出的算法可以在不增加算法复杂度的情况下,有效提高视频的主客观质量。     

High‐Performance Spatial and Temporal Error‐Concealment Algorithms for Block‐Based Video Coding Techniques

A compressed video bitstream is sensitive to errors that may severely degrade the reconstructed images even when the bit error rate is small. One approach to combat the impact of such errors is the use of error concealment at the decoder without increasing the bit rate or changing the encoder. For spatial‐error concealment, we propose a method featuring edge continuity and texture preservation as well as low computation to reconstruct more visually acceptable images. Aiming at temporal error concealment, we propose a two‐step algorithm based on block matching principles in which the assumption of smooth and uniform motion for some adjacent blocks is adopted. As simulation results show, the proposed spatial and temporal methods provide better reconstruction quality for damaged images than other methods.     

Cell-loss concealment techniques for layered video codecs in an ATMnetwork

A layered video coding scheme with its inherent cell loss resilience has been considered as a means for transporting reliably integrated video services over an asynchronous transfer mode (ATM) based network such as the broadband-ISDN. This paper presents some data concealment techniques that can be implemented in the coding of video data at the encoder, in the ATM adaptation layer (AAL) functionality of network realization and at the decoder to improve the performance of a layered codec under different conditions of video packet loss. The performance of these techniques are verified by software simulations.     

MPEG-2视频信号错误控制和错误掩盖技术

以ＭＰＥＧ－２的语法规范作基础,从控制错误发生和控制错误传播的角度,简述了ＭＥＰＧ－２的分级工具和视频码流层结构,比较详细地介绍了Ｉ帧、Ｐ帧和Ｂ帧的错误掩盖技术,此外,还推荐了一些错误掩盖方法。     

视频通信中的误码掩盖技术初探

在数字视频通信中,由于信道不理想等原因容易产生误码,而且误码在空间和时间上的扩散会导致重建图像质量不同程度的下降。最近发展起来的误码掩盖技术就是用来解决这类问题、改进重建视频质量的重要方法之一。在简要介绍误码掩盖的基本原理和相关技术的基础上,着重分析了新近采用的几种误码掩盖方法的应用范围和性能,并指出了它们的发展前景。．     

Joint source-channel coding for wireless object-based video communications utilizing data hiding.

In recent years, joint source-channel coding for multimedia communications has gained increased popularity. However, very limited work has been conducted to address the problem of joint source-channel coding for object-based video. In this paper, we propose a data hiding scheme that improves the error resilience of object-based video by adaptively embedding the shape and motion information into the texture data. Within a rate-distortion theoretical framework, the source coding, channel coding, data embedding, and decoder error concealment are jointly optimized based on knowledge of the transmission channel conditions. Our goal is to achieve the best video quality as expressed by the minimum total expected distortion. The optimization problem is solved using Lagrangian relaxation and dynamic programming. The performance of the proposed scheme is tested using simulations of a Rayleigh-fading wireless channel, and the algorithm is implemented based on the MPEG-4 verification model. Experimental results indicate that the proposed hybrid source-channel coding scheme significantly outperforms methods without data hiding or unequal error protection.