一种基于率失真优化的关键参考帧选择算法

为抑制压缩视频流的传输错误扩散,提出了一种新的基于关键帧的无反馈参考帧选择算法,通过采用一种适用各种掩盖策略的端到端失真度估计方法,评价出选择不同参考帧情况下的解码器失真度期望,在率失真框架内选择最合适的参考帧,使解码端的失真度达到最小.基于H. 263+ TMN8测试模型和IP包丢失的模拟实验表明,提出的率失真优化关键参考帧算法比传统的预测编码方法,PSNR值大约提高了1~3dB.当数据丢包率严重时,优势尤为明显.     

基于多视点视频编码的差错控制算法

 多视点视频编码(Multiview Video Coding,MVC)利用运动估计和视差估计取得了较好的编码性能,但在易错的网络环境下传输MVC视频码流,将导致差错在视点内与视点间进行扩散.针对多视点视频的编码特性,提出了一种端到端的失真度估计模型,并将此模型与率失真优化相结合得到一种基于联合信源信道的编码模式选择算法.实验结果表明该方法能够在易错网络环境下有效的提高多视点视频的传输效率.     

基于反馈的自适应参考帧选择的率失真优化分析

为了提高数字视频实时传输的容差能力,该文提出了一种基于反馈的自适应参考帧选择方法。在传统参考帧选择优点的基础上,设计了一种新的通过率失真技术优化选择参考帧的帧间帧预测结构,分析了选择不同参考帧编码时的编码码率以及它对编码失真的影响。引入了信道代价均值的概念,探讨了信道误差的扩散过程及其对信道失真的影响。在联合信源和信道的率失真优化框架内,选择使得率失真代价最小的候选参考帧作为当前帧编码的参考帧,有效地改善了视频传输质量,并减少了编/解码器所需缓存的参考帧。仿真实验显示自适应参考帧选择的PSNR性能在不同码率和丢包率下均优于经典的ACK和NACK参考帧选择方法。     

Optimizing Motion Compensated Prediction for Error Resilient Video Coding

This paper is concerned with optimization of the motion compensated prediction framework to improve the error resilience of video coding for transmission over lossy networks. First, accurate end-to-end distortion estimation is employed to optimize both motion estimation and prediction within an overall rate-distortion framework. Low complexity practical variants are proposed: a method to approximate the optimal motion via simple distortion and source coding rate models, and a source-channel prediction method that uses the expected decoder reference frame for prediction. Second, reference frame generation is revisited as a problem of filter design to optimize the error resilience versus coding efficiency tradeoff. The special cases of leaky prediction and weighted prediction (i.e., finite impulse response filtering), are analyzed. A novel reference frame generation approach, called ?generalized source-channel prediction?, is proposed, which involves infinite impulse response filtering. Experimental results show significant performance gains and substantiate the effectiveness of the proposed encoder optimization approaches.     

基于ZIG-ZAG交织的H.264/AVC容错编码算法

本文提出了一种基于ZIG-ZAG交织的灵活宏块排列算法ZFMO,按ZIG-ZAG扫描方式沿次对角线方向上交织排列宏块,从而在低位率下达到了编码效率和容错性能的最佳平衡点.实验结果表明,ZFMO在低丢包率的网络环境下比当前H.264/AVC的推荐算法具有更好的编码效率和容错性能.经过率失真优化后的ZFMO算法性能进一步提高.     

Rate-Distortion Optimized Motion-Compensated Prediction for Packet Loss Resilient Video Coding

A rate-distortion optimized motion-compensated prediction method for robust video coding is proposed. Contrasting methods from the conventional literature, the proposed approach uses the expected reconstructed distortion after transmission, instead of the displaced frame difference in motion estimation. Initially, the end-to-end reconstructed distortion is estimated through a recursive per-pixel estimation algorithm. Then the total bit rate for motion-compensated encoding is predicted using a suitable rate distortion model. The results are fed into the Lagrangian optimization at the encoder to perform motion estimation. Here, the encoder automatically finds an optimized motion compensated prediction by estimating the best tradeoff between coding efficiency and end-to-end distortion. Finally, rate-distortion optimization is applied again to estimate the macroblock mode. This process uses previously selected optimized motion vectors and their corresponding reference frames. It also considers intraprediction. Extensive computer simulations in lossy channel environments were conducted to assess the performance of the proposed method. Selected results for both single and multiple reference frames settings are described. A comparative evaluation using other conventional techniques from the literature was also conducted. Furthermore, the effects of mismatches between the actual channel packet loss rate and the one assumed at the encoder side have been evaluated and reported in this paper     

Rate control for real-time video network transmission on end-to-end rate-distortion and application-oriented QoS

In the merged multimedia packet-switched networks, the limited or no end-to-end Quality of Service (QoS) guarantees induce that rate control has been evolved to joint source-channel adjustment architecture based on application-oriented QoS. Based on the statistical analysis under the spatial intra and temporal inter prediction subject to a universal spatial-temporal coding framework and a general error concealment by the decoder, an end-to-end distortion estimation model is proposed. On the basis of the analytic model, this paper fulfills a picture quality parameters selection solution with rate-distortion (R-D) Lagrange optimization for a general coding engine including H.264/AVC, exploiting either the source-driven iterative prediction or feedback recursion. Further, a corresponding joint source-channel rate control strategy is proposed. For the real-time variable bit-rate (VBR) video transmission under a given time-varying network condition, the strategy could estimate an instantaneous available transmission rate on traffic smoothing and codec's buffer control, adopt the proposed an end-to-end distortion regressive model and a global optimal error control parameters selection, and address the consistent bit allocation in group of picture (GOP) level, picture level, and MB level. The extensive network simulation experiments show a better and more consistent end-to-end picture quality, in contrast with the locally optimal control strategy at MB level.     

A joint encoder–decoder error control framework for stereoscopic video coding

Stereoscopic video coding (SSVC) plays an important role in various 3D video applications. In SSVC, robust stereoscopic video transmission over error-prone networks is still a challenge problem to be solved. In this paper, we propose a joint encoder–decoder error control framework for SSVC, where error-resilient source coding, transmission network conditions, and error concealment scheme are jointly considered to achieve better error robustness performance. The proposed joint encoder–decoder error control framework includes two parts: an error concealment algorithm at the decoder side and a rate–distortion optimized error resilience algorithm at the encoder side. For error concealment at the decoder side, an overlapped block motion and disparity compensation based error concealment scheme is proposed to adaptively utilize inter-view correlations and temporal correlations. For error resilience at the encoder side, first, the inter-view refreshment is proposed for SSVC to suppress error propagations. Then, an end-to-end distortion model for SSVC is derived, which jointly considers the transmission network conditions, inter-view refreshment, and error concealment tools at the decoder side. Finally, based on the derived end-to-end distortion model, the rate–distortion optimized error resilience algorithm is presented to adaptively select inter-view, inter- or intra-coding for SSVC. The experimental results show that the proposed joint encoder–decoder error control framework has superior error robustness performance for stereoscopic video transmission over error-prone networks.     

Error concealment for SNR scalable video coding

This paper proposes an efficient error concealment method for SNR scalable coded video. The algorithm adaptively selects a proper concealment candidate from the base or the enhanced pictures to conceal the artifact of a lost enhancement block. To determine the best concealment candidate, we propose a trial process in which the concealment candidates are examined based on two criteria: (1) picture continuity at the border of concealed macroblocks, and (2) to satisfy the coding distortion bound of the base layer coefficients when they are available. For the latter, requantization of the concealed picture with the base layer quantizer step size and its dequantized pixels should result in zero distortion. We have implemented the method on a proposed SNR scalable H.264 video codec and compared the decoded video quality against just copying the base layer pixels into the enhanced picture. Simulation results show that the proposed method can achieve a considerable improvement by up to 3 dB especially in situations where the enhancement layer contains a large portion of the picture information. This will make scalable video transmission more successful over unreliable channels.     

Prediction of transmission distortion for wireless video communication: Algorithm and application

In this paper, we develop algorithms for estimating transmission distortion in wireless video communication systems. By leveraging the analytical results obtained in our previous works, we design low complexity algorithms that are capable of estimating transmission distortion accurately. We also extend our algorithm for pixel-level transmission distortion estimation to pixel-level end-to-end distortion estimation. Furthermore, we apply our pixel-level end-to-end distortion estimation algorithm to prediction mode decision in H.264 encoder. Experimental results show that (1) our transmission distortion estimation algorithm is more accurate and more robust against inaccurate channel estimation than existing distortion estimation algorithms; (2) our mode decision algorithm achieves remarkable PSNR gain over the existing algorithms for prediction mode decision in H.264 encoder, e.g., an average PSNR gain of 1.44 dB for ‘foreman’ sequence when Packet Error Probability (PEP) equals 5%.     

An integrated temporal error concealment for H.264/AVC based on spatial evaluation criteria

Owing to error-prone transmission networks, the compressed video bit stream is prone to packet loss in the transmission channel. This loss causes serious distortion and the distortion will propagate to successive frames, especially in highly compressed video coding standard. Therefore, it is very important to efficiently enhance the restored result. In this paper, an integrated temporal error concealment technique for H.264/AVC is proposed. The technique could effectively restore the corrupted data by adaptively integrating error concealment approaches with the adaptive weight-based switching algorithm. The integrated mechanism is based on spatial evaluation criteria, judged by boundary distortion estimation and texture intensity. Experimental results show that the technique could effectively enhance the performance of error concealment.     

可变尺寸块运动估计的R—D优化

本文应用率失真理论在固定码率的前提下对可变尺寸块运动估计过程进行R－D优化,获得了最佳预测块划分和最小运动补偿误差,并提出了基于树形结构的运动矢量编码方案。R－D优化问题的求解通过引入失真度－码率梯度,将R－D优化与分层运动估计结合在一起,克服了GP－BFOS算法需要对各个分块层次进行运动估计及所得局部最优解严格位于R－D曲线凸包络线上的弊病。软件模拟结果显示本算法大大降低了G－BFOS优化算法的计算量,且运动补偿RSNR值高于G－BFOS算法结果。     

Error-resilient image and video transmission over the Internet using unequal error protection

This paper presents a new bit-plane-wise unequal error protection algorithm for progressive bitstreams transmitted over lossy networks. The proposed algorithm protects a compressed embedded bitstream generated by a 3-D SPIHT algorithm by assigning an unequal amount of forward error correction (FEC) to each bit-plane. The proposed algorithm reduces the amount of side information needed to send the size of each code to the decoder by limiting the number of quality levels to the number of bit-planes to be sent while providing a graceful degradation of picture quality as packet losses increase. We also apply our proposed algorithm to transmission of JPEG 2000 coded images over the Internet. To get additional error-resilience at high packet loss rates, we extend our algorithm to multiple-substream unequal error protection. Simulation results show that the proposed algorithm is simple, fast and robust in hostile network conditions and, therefore, can provide reasonable picture quality for video applications under varying network conditions.     

一种有效的MIMO-OFDM系统MAP信道估计

对于MIMO-OFDM系统,最大后验概率(MAP)信道估计算法可通过期望最大化(EM)算法降低计算复杂度,但将产生误差平底(error floor)现象。并且,系统的数据传输效率受限于发送端天线的数目。针对这些问题,该文提出了一种有效的MAP信道估计算法,并分析了算法的性能。所提算法在利用EM算法减小MAP 算法计算复杂度的基础上,利用角域内信道间的独立性降低估计误差。为改善系统数据传输效率及估计性能,通过多个OFDM符号进行联合的信道估计。仿真实验验证了所提算法拥有更好的估计性能和数据传输效率。     

Accurate Distortion Estimation and Optimal Bandwidth Allocation for Scalable H.264 Video Transmission Over MIMO Systems

In this paper, we propose an optimal strategy for the transmission of scalable video over packet-based multiple-input multiple-output (MIMO) systems. The scalable extension of H.264/AVC that provides a combined temporal, quality and spatial scalability is used. For given channel conditions, we develop a method for the estimation of the distortion of the received video and propose different error concealment schemes. We show the accuracy of our distortion estimation algorithm in comparison with simulated wireless video transmission with packet errors. In the proposed MIMO system, we employ orthogonal space-time block codes (O-STBC) that guarantee independent transmission of different symbols within the block code. In the proposed constrained bandwidth allocation framework, we use the estimated end-to-end decoder distortion to optimally select the application layer parameters, i.e., quantization parameter (QP) and group of pictures (GOP) size, and physical layer parameters, i.e., rate-compatible turbo (RCPT) code rate and symbol constellation. Results show the substantial performance gain by using different symbol constellations across the scalable layers as compared to a fixed constellation.      

一种抗误码能力强的无线信道图像传输方案

本文提出一种新的综合源编码和信道编码的无线信道图像传输方案，对子波变换后各个子带采用基于统计特性的变系数定长（VCFL）编码，数据经过Reed-Solomon编码后选择不同码率的码率兼容删除卷积码（RCPC）编码以提供不同程度的差错保护（UEP），最后数据经交织后传输。在给定传输的总比特数的情况下，通过优化算法使总体失真最小。模拟结果表明新方案在无线衰落信道上表现出良好的性能。     

An Adaptive Control for the Propagation Errors Incurred by DCT Coefficient‐Dropping Transcoder

This paper presents a new distortion control scheme with a simple estimation model for the propagation errors incurred by dropping some parts of the bitstream in a frame dropping–coefficient dropping (FD‐CD) transcoder. The primary goal of this paper is to facilitate bit‐rate conversions and rate‐distortion controls in the compressed domain without introducing a full decoding and re‐encoding system in the pixel domain. First, the error propagation behavior over several frame sequences due to coefficient dropping is investigated on the basis of statistical and empirical properties. Then, such properties are used to develop a simple estimation model for the CD distortion accounting for the characteristics of the underlying coded‐frame. Finally, the proposed estimation model allows us to determine the amount of coefficient dropping and to effectively allocate rate‐distortions into coded‐frames. Experimental results show that the proposed estimation model accurately describes the characteristics of propagation errors adaptively in the compressed domain and can be easily applied to distortion control over different kinds of video sequences.     

一种低复杂度的多参考帧选择率失真优化方法

视频在无线易错信道上传输,报文丢失会导致图像质量显著下降。多参考帧选择方法可以降低误码扩散,从而提高误码信道上传输视频的图像质量。但是参考帧选择方法也使得编码器计算复杂度大大增加。基于多参考帧选择技术提出一种低复杂度的率失真优化方法。该方法首先在编码器上重建误码图像,使得编码器和解码器上的参考图像一致,然后利用端到端率失真优化模型,在重建的误码图像与编码图像的上一帧图像中选择最优的参考帧。在进行率失真计算时,采用线性预测方法计算运动补偿后图像差值的方差,减少了运动估计运算,从而大大降低计算复杂度。实验结果表明相对于帧内刷新方法该方法显著提高图像质量,相对于一般多参考帧选择方法,在损失很小性能的情况下,大大降低了计算复杂度。     

Video coding with optimal inter/intra-mode switching for packetloss resilience

Resilience to packet loss is a critical requirement in predictive video coding for transmission over packet-switched networks, since the prediction loop propagates errors and causes substantial degradation in video quality. This work proposes an algorithm to optimally estimate the overall distortion of decoder frame reconstruction due to quantization, error propagation, and error concealment. The method recursively computes the total decoder distortion at pixel level precision to accurately account for spatial and temporal error propagation. The accuracy of the estimate is demonstrated via simulation results. The estimate is integrated into a rate-distortion (RD)-based framework for optimal switching between intra-coding and inter-coding modes per macroblock. The cost in computational complexity is modest. The framework is further extended to optimally exploit feedback/acknowledgment information from the receiver/network. Simulation results both with and without a feedback channel demonstrate that precise distortion estimation enables the coder to achieve substantial and consistent gains in PSNR over known state-of-the-art RD- and non-RD-based mode switching methods     

Rate control of MPEG video for consistent picture quality

We propose a rate control scheme using a rate-distortion (R-D) estimation model, which produces a consistent picture quality between consecutive frames. Our R-D estimation method offers a closed-form mathematical model that enables us to predict the bits and the distortion generated from a frame encoded at a given quantization parameter (QP) and vice versa. Its most attractive feature is its low computational complexity. Furthermore, it is accurate enough to be applied to practical video coding. In our simulation, the estimation errors for rate and distortion are less than 2.5% and 1.5%, respectively. Therefore, the proposed rate control scheme is appropriate for applications requiring low delay, low complexity, and the ability to control output bit-rate and quality accurately. Our scheme ensures that the video buffers do not underflow or overflow by satisfying the buffer constraint, and it also prevents quality difference between consecutive frames from exceeding a certain demanded level by adopting a distortion constraint. In addition, a consistent picture quality is maintained within a frame, and error propagation, caused by quality degradation of anchor frames, is reduced by differentiating the control procedure for anchor frames from that for nonanchor frames. Simulation results show that our control scheme achieves 0.52-1.84 dB peak signal-to-noise ratio (PSNR) gain over MPEG-2 Test Model 5 (TM5) rate control and maintains very consistent quality within a frame as well as between frames.