Novel prediction schemes for error resilient video coding

A conventional video codec uses encoder reconstruction of previous frames for motion compensated prediction. This is designed to minimize the encoder prediction error and assumes error free transmission. In this paper we use a modified prediction mechanism both at the encoder and decoder and propose techniques to improve the error resilience of H.264/AVC when transmitted over error prone networks. In our schemes we provide greater emphasis on Intra pixels during the formation of the reference frame used for prediction, thereby achieving better resilience. We also incorporate leaky prediction to further improve the robustness. We apply leaky prediction selectively at a macroblock level based on a simple mean square error metric in order to reduce the bit-rate penalty. Substantial performance gains have been observed in simulations. The effectiveness of using leaky prediction can be observed in medium and fast moving video sequences.     

Efficient traffic prediction scheme for real-time VBR MPEG videotransmission over high-speed networks

The efficient transportation of real-time variable bit rate (VBR) video traffic in high-speed networks is currently an active area of research. The capability to predict VBR video traffic can significantly improve the effectiveness of numerous management tasks, including dynamic bandwidth allocation and congestion control. This paper proposes an adaptive traffic prediction method for VBR MPEG videos, a major multimedia application. Rapid traffic variations due to scene changes are analyzed, then a prediction scheme using the identification of scene changes related to I and P frames is presented. For predicting multiplexed MPEG traffic, a prediction interval is derived that represents a highly correlated traffic sequence. In addition, to reduce the prediction error, a less fluctuating signal instead of the original multiplexed traffic is used as the input for the predictor. Simulation results show that the proposed method is able to predict the original traffic more accurately than the conventional LMS method     

智能集成VBR MPEG视频流量预测模型

作为数字媒体网络视频通信的主要方式,VBR MPEG视频流量的预测能力是直接关系缓冲区设计、动态带宽分配及拥塞控制等提高网络服务质量的关键因素.因此针对MPEG视频流的复杂特性,充分利用人工智能方法的优势,提出并建立了基于模糊神经网络的智能集成VBR MPEG 视频流量预测模型.采用模糊预测模型提高预测精度,利用神经网络解决预测的实时性问题.实验结果表明,与标准AR预测模型相比,该模型预测的准确度和可靠性显著提高,且算法简单易于推广到其他方法中使用.     

基于多假设运动补偿去噪的迭代边信息改进算法

在分布式视频压缩系统中,边信息的质量对编码效率有至关重要的作用。利用已解码图像信息提高边信息质量是近年来研究的热点之一。根据边信息和已解码信息与原始信息之间的噪声关系,该文提出了一种基于多假设运动补偿去噪的迭代边信息改进算法。首先对原始边信息进行多假设运动补偿去噪,生成质量更好而不仅是相似的补偿图像,然后利用每个码平面的解码信息对边信息进行迭代改进。实验结果表明,该算法能减少码流,显著提高Wyner-Ziv (WZ)帧质量,从而有效的改善分布式视频压缩的率失真性能。     

Proposed dynamic error control techniques for QoS improvement of wireless 3D video transmission

Error control techniques like error resilience (ER) and error concealment (EC) are efficient techniques to ameliorate the lost macroblocks (MBs) in the 3D video (3DV) communication system. In this paper, we propose efficient and adaptive hybrid ER‐EC algorithms for 3DV transmission over error‐prone wireless channels. At the encoder, adaptive preprocessing ER mechanisms are proposed through using the context adaptive variable length coding entropy, slice structured coding modes, and explicit flexible macroblock ordering mapping. They are used to assist the suggested EC techniques at the decoder to accurately reconstruct the erroneous MBs and frames. At the decoder, an efficient postprocessing EC technique with multiproposition methods is proposed to dynamically select the convenient EC hypothesis method based on the size of the lost MBs, the faulty view, and the frame type. It conceals the received erroneous MBs of intra‐encoded and inter‐encoded frames of the transmitted 3DV by exploiting the temporal, spatial, and inter‐view correlations among frames and views. To further improve the decoded 3DV quality, a weighted overlapping block motion and disparity compensation technique is used to reinforce the performance of the suggested ER‐EC techniques. Experimental results on various 3DV streams prove that the suggested techniques have considerably acceptable subjective and objective 3DV performance. They achieve an improved average peak signal‐to‐noise ratio gain by almost 2.85 dB compared to the conventional error control algorithms at a packet loss rate = 40%.     

基于HEVC帧内预测的复杂度控制

帧内预测在视频编码中是非常重要的模块.在视频实时编码与传输过程中,场景切换会经常出现.此时,一般会采用全Ⅰ帧编码.研究发现,即使是全Ⅰ帧编码,也往往会非常耗时.基于编码单元深度范围和帧内预测中候选预测方向个数研究了HEVC编码器的复杂度控制问题.针对不同的目标编码复杂度,算法自适应地选择不同的方法来优化编码过程.实验结果表明,该算法在保证视频质量的前提下实现了对不同复杂度目标的控制.     

Error recovery for interactive video transmission over the Internet

Real-time interactive video transmission in the current Internet has mediocre quality because of high packet loss rates. Loss of packets in a video frame manifests itself not only in the reduced quality of that frame but also in the propagation of that distortion to successive frames. This error propagation problem is inherent in any motion compensation-based video codec. In this paper, we present a new error recovery scheme, called recovery from error spread using continuous updates (RESCU), that effectively alleviates error propagation in the transmission of interactive video. The main benefit of the RESCU scheme is that it allows more time for transport-level recovery such as retransmission and forward error correction to succeed while effectively masking out delays in recovering lost packets without introducing any playout delays, thus making it suitable for interactive video communication. Through simulation and real Internet experiments, we study the effectiveness and limitations of our proposed techniques and compare their performance to that of existing video error recovery techniques including H.263+ (NEWPRED). The study indicates that RESCU is effective in alleviating the error spread problem and can sustain much better video quality with less bit overhead than existing video error recovery techniques under various network environments     

基于区域划分的立体视频整帧丢失错误隐藏新算法

针对立体视频传输中右视点整帧丢失,提出了一种基于区域划分的立体视频整帧丢失错误隐藏新算法。首先通过丢失帧视点内和视点间相邻帧的运动信息,估计丢失帧的运动区域与静止区域;然后利用视点间的全局视差,将丢失帧运动区域进一步划分为遮挡边界区域和非遮挡边界区域,并对遮挡边界区域和非遮挡边界区域分别采用运动补偿预测和视差补偿预测进行恢复,而对静止区域则直接采用帧拷贝的方法恢复丢失信息。实验结果表明,本文算法有效提高了所恢复的立体视频丢失帧图像的主客观质量。     

Perceptual-based distributed video coding

In this paper, we propose a perceptual-based distributed video coding (DVC) technique. Unlike traditional video codecs, DVC applies video prediction process at the decoder side using previously received frames. The predicted video frames (i.e., side information) contain prediction errors. The encoder then transmits error-correcting parity bits to the decoder to reconstruct the video frames from side information. However, channel codes based on i.i.d. noise models are not always efficient in correcting video prediction errors. In addition, some of the prediction errors do not cause perceptible visual distortions. From perceptual coding point of view, there is no need to correct such errors. This paper proposes a scheme for the decoder to perform perceptual quality analysis on the predicted side information. The decoder only requests parity bits to correct visually sensitive errors. More importantly, with the proposed technique, key frames can be encoded at higher rates while still maintaining consistent visual quality across the video sequence. As a result, even the objective PSNR measure of the decoded video sequence will increase too. Experimental results show that the proposed technique improves the R-D performance of a transform domain DVC codec both subjectively and objectively. Comparisons with a well-known DVC codec show that the proposed perceptual-based DVC coding scheme is very promising for distributed video coding framework.     

Error-resilient video transmission using long-term memorymotion-compensated prediction

Long-term memory prediction extends the spatial displacement vector utilized in hybrid video coding by a variable time delay, permitting the use of more than one reference frame for motion compensation. This extension leads to improved rate-distortion performance. However, motion compensation in combination with transmission errors leads to temporal error propagation that occurs when the reference frames at the coder and decoder differ. In this paper, we present a framework that incorporates an estimated error into rate-constrained motion estimation and mode decision. Experimental results with a Rayleigh fading channel show that long-term memory prediction significantly outperforms the single-frame prediction H.263-based anchor. When a feedback channel is available, the decoder can inform the encoder about successful or unsuccessful transmission events by sending positive (ACK) or negative (NACK) acknowledgments. This information is utilized for updating the error estimates at the encoder. Similar concepts, such as the ACK and NACK mode known from the H.263 standard, are unified into a general framework providing superior transmission performance     

Essential Information Aggregation of intra coded video frame for wireless transmisson

A novel error resilient method based on Essential Information Aggregation （EIA） technique is proposed for robust wireless transmission ofintra coded video frames. This method modifies the original video stream structure so as to mitigate the effect of decoding loss of synchronization within a video frame, for which Macro Block Aligned （MBA） error detection method is accompanied to be developed. Experiments on an Additive White Gaussian Noise （AWGN） wireless simulation platform and H.263＋ baseline video codec show that it can obtain better subjective and objective image quality than existing error resilience approaches.     

Frame distortion estimation for unequal error protection methods in scalable video coding (SVC)

In many multimedia applications, coded video is transmitted over error prone heterogeneous networks. Because of the predictive mechanism used in video coding, transmission error would propagate temporally and spatially and would result in significant quality losses. In order to address this problem, different error resilience methods have been proposed. One of the techniques, which is commonly used in video streaming, is unequal error protection (UEP) of scalable video coding (SVC). In this technique, different independent layers of an SVC stream are protected differently and based on their importance by using forward error correction (FEC) codes. Accurately analyzing the importance or utility of each video part is a critical component and would lead to a better protection and higher quality of the received video. Calculation of the utility is usually based on multiple decoding of sub-bitstreams and is highly computationally complex. In this work, we propose an accurate low complexity utility estimation technique that can be used in different applications. This technique estimates the utility of each network abstraction layer (NAL) by considering the error propagation to future frames. We utilize this method in an UEP framework with the scalable extension of H.264/AVC codec and it achieves almost the same performance as highly complex estimation techniques (an average loss of 0.05 dB). Furthermore, we propose a low delay version of this technique that can be used in delay constrained application. The estimation accuracy and performance of our proposed technique are studied extensively.     

高效率的多视点视频编码预测结构*

预测结构是多视点视频编码(Multi-View Video Coding,MVC)研究的主要内容之一。MVC目前采用HHI(Heinrich-Hertz-Institute)提出的分层次B帧预测结构(HBP),比联播预测结构获得了更好的压缩效率。分析了多种预测结构,并针对平行摄像机采集的多视点视频序列,提出了一种新的预测结构AS_EIPP,该结构充分利用相邻视点间的相关性和多参考帧模式,进一步提高了压缩效率。在多视点视频软件测试平台JMVC8.3上进行验证,实验结果表明:新的预测结构在保证重建视频质量基本不变的前提下,压缩效率比HBP预测结构提高了1%～4%。     

Content analysis based smart macroblock rearrangement for error resilience in wireless video transmission

Transmission of compressed video is very sensitive to channel error. The use of resynchronization marks is an efficient method in improving the error resilient performance of a video stream in error-prone environment. In this paper, a smart macroblock rearrangement (SMR) method is proposed to enhance the performance of the resynchronization mark insertion technique for intracoded frames in wireless video transmission. The proposed method makes use of content analysis to rearrange and encode macroblocks between adjacent resynchronization marks. Experiments using the SMR method in both H.263+ and H.264 codecs show that the proposed method outperforms some existing algorithms in both PSNR performance and visual quality.     

Adaptive reversible video watermarking based on motion-compensated prediction error expansion with pixel selection

In this study, a motion-compensated prediction error expansion-based adaptive reversible video watermarking algorithm is proposed. Blocks of motion-compensated frames are classified as smooth and non-smooth according to their prediction errors. Unlike the current reversible video watermarking methods that apply a single watermarking strategy to all blocks, the proposed method uses two different strategies for smooth and non-smooth blocks. This adaptive strategy is shown to increase watermarking capacity. In addition, an approach is suggested to detect those pixels causing high distortion in the watermarked video and they are not used in watermarking to limit the distortion occurring in the original video. Simulations show that the proposed method is superior to existing methods in terms of capacity and distortion.     

基于运动变化信息熵的视频序列时域亚采样技术

视频序列的时域亚采样对于低比特率信道上的视频传输和存储空间受限条件下的视频存储具有十分重要的意义。目前普遍采用的时域等间隔亚采样方法有时会造成视频序列中重要的运动变化信息的丢失。针对这一情况,本文提出了基于运动变化信息熵的视频序列时域亚采样技术,通过帧间运动场分析,利用运动场总体能量和运动补偿后的残差能量综合描述视频序列的运动变化信息,并根据信息熵最大化准则确定各个时域亚采样时刻点。大量的对比实验证明该方法明显优于时域等间隔亚采样方法,它较完整地反映了序列图像的运动变化过程,更有利于对视频内容的理解。     

A video error concealment method using data hiding based on compressed sensing over lossy channel

The video error concealment with data hiding (VECDH) method aims to conceal video errors due to transmission according to the auxiliary data directly extracted from the received video file. It has the property that can well reduce the error propagated between spatially/temporally correlated macro-blocks. It is required that, the embedded information at the sender side should well capture/reflect the video characteristics. Moreover, the retrieved data should be capable of correcting video errors. The existing VECDH algorithms often embed the required information into the corresponding video frames to gain the transparency. However, at the receiver side, the reconstruction process may loss important information, which could result in a seriously distorted video. To improve the concealment performance, we propose an efficient VECDH algorithm based on compressed sensing (CS) in this paper. For the proposed method, the frame features to be embedded in every video frame are generated from the frame residuals CS measurements and scrambled with other frame features as marked data. The marked data is embedded into the corresponding frames by modulating color-triples for its least impacts on the carriers. For the receiver, the extracted data is used to reconstruct residuals to conceal errors. Error positions are located using the set theory. Since the CS has the ability to sample a signal within a lower sampling rate than the Shannon–Nyquist rate, the original signal could be reconstructed very well in theory. This indicates that the proposed method could benefit from the CS, and therefore keep better error concealment behavior. The experimental results show that the PSNR values gain about 10 dB averagely and the proposed scheme in this paper improves the video quality significantly comparing with the exiting VECDH schemes.     

Non-predictive multi-scaled based MLVQ video coding for robust transmission over mobile channels

A new non-predictive video codec for mobile applications is presented. The scheme omits the prediction step in the temporal axis and increases robustness of its transmission in the mobile channel. The same subbands from each frame in the group of frames (GOP) are joined together to exploit their spatial-temporal redundancies. The significant vectors of the joined subbands within the GOP are quantised using a novel multi-stage lattice vector quantisation. This process reduces quantisation errors and enhances the reconstructed frame quality. Experimental results are shown to be significantly better than H.263 and comparable to the current H.264 standards in an erroneous hilly terrain mobile environment using the TETRA channel simulator for some test video sequences.     

Reference picture selection with decreased temporal dependency for HEVC error resilience

The high level of compression efficiency achieved by the High-Efficiency Video Coding standard (HEVC) decreases the robustness of the encoded bitstreams. This increased susceptibility to network errors leads to end video quality degradation. Moreover, due to the high computational complexity of HEVC, high-resolution video transmission with time constraints over hostile channels such as wireless networks becomes more challenging. This paper proposes a reference picture selection-based error-resilient method to reduce the temporal error propagation due to high-trip delay and frame-copy concealment error. First, the encoder selects the reference pictures based on the error status received from the feedback channel, taking into consideration the Rate-Distortion-Optimization (RDO). Second, the temporal information mismatch prediction resulting from the error concealment is reduced by decreasing the temporal dependency between adjacent frames based on new motion-estimation tools. Results show a PSNR gain of about 6.13 dB, 5.20 dB and 4.72 dB for 1080p, 720p and 480p resolutions respectively.     

面向三维高效视频编码的深度图错误隐藏

基于多视点视频序列视点内、视点间存在的相关性,并结合视点间运动矢量共享技术,该文提出一种面向3维高效视频编码中深度序列传输丢包的错误隐藏算法。首先,根据3D高效视频编码(3D-HEVC)的分层B帧预测(HBP)结构和深度图纹理特征,将深度图丢失块分成运动块和静止块;然后,对于受损运动块,使用结合纹理结构的外边界匹配准则来选择相对最优的运动/视差矢量进行基于位移矢量补偿的错误掩盖,而对受损静止块采用参考帧直接拷贝进行快速错误隐藏;最后,使用参考帧拆分重组来获取新的运动/视差补偿块对修复质量较差的重建块进行质量提升。实验结果表明：相较于近年提出的对比算法,该文算法隐藏后的深度帧平均峰值信噪比(PSNR)能提升0.25～2.03 dB,结构相似度测量值(SSIM)能提升0.001～0.006,且修复区域的主观视觉质量与原始深度图更接近。