Rate control strategy for embedded wavelet video coders

An investigation into the use of rate-control strategies for embedded wavelet video encoders is presented. It is shown that the best model for the rate×distortion (R-D) curve of these encoders is piecewise linear. Also, an effective iterative procedure is proposed for dealing with the problem of frame dependency, which yields improved rate×distortion results     

Operational rate-distortion modeling for wavelet video coders

Based on our statistical investigation of a typical three-dimensional (3-D) wavelet codec, we present a unified mathematical model to describe its operational rate-distortion (RD) behavior. The quantization distortion of the reconstructed video frames is assessed by tracking the quantization noise along the 3-D wavelet decomposition trees. The coding bit-rate is estimated for a class of embedded video coders. Experimental results show that the model captures sequence characteristics accurately and reveals the relationship between wavelet decomposition levels and the overall RD performance. After being trained with offline RD data, the model enables accurate prediction of real RD performance of video codecs and therefore can enable optimal RD adaptation of the encoding parameters according to various network conditions.     

一种时域质量稳定的三维小波视频编码率控制技术

三维小波视频编码提供了分辨率、时域和质量的可伸缩性。在三维小波编码中，如何调整各小波频带间的码率分配以达到最佳的码率控制至关重要。就小波变换和反变换的结构而言，常规的三维小波视频编码方法会导致解码视频帧沿时间方向播放质量的周期振荡，这种周期的振荡会影响了观看质量。本文研究了重建信号与小波频带间失真分布的关系，提出了一种具有稳定的时域解码质量的率控制方法，实验结果表明该算法在保证播放质量稳定的同时，还具有与常规三维小波视频编码方法近似的平均解码质量。     

A Fast Intra Skip Detection Algorithm for H.264/AVC Video Encoding

A fast intra skip detection algorithm based on the rate‐distortion (RD) cost for an inter frame (P‐slices) is proposed for H.264/AVC video encoding. In the H.264/AVC coding standard, a robust rate‐distortion optimization technique is used to select the best coding mode and reference frame for each macroblock (MB). There are three types of intra predictions according to profiles. These are 16×16 and 4×4 intra predictions for luminance and an 8×8 intra prediction for chroma. For the high profile, an 8×8 intra prediction has been added for luminance. The 4×4 prediction mode has 9 prediction directions with 4 directions for 16×16 and 8×8 luma, and 8×8 chrominance. In addition to the inter mode search procedure, an intra mode search causes a significant increase in the complexity and computational load for an inter frame. To reduce the computational load of the intra mode search at the inter frame, the RD costs of the neighborhood MBs for the current MB are used and we propose an adaptive thresholding scheme for the intra skip extraction. We verified the performance of the proposed scheme through comparative analysis of experimental results using joint model reference software. The overall encoding time was reduced up to 32% for the IPPP sequence type and 35% for the IBBPBBP sequence type.     

一种无帧内失真漂移的HEVC视频自适应隐写算法

随着数字视频的普及,HEVC(high efficiency video coding)视频隐写受到了越来越多的关注。残差系数域隐写具有高安全性和低码率增长的特点,然而,视频编码过程中重构的系数不满足块间相互独立的性质,导致修改视频残差系数往往会带来较为严重的失真漂移。本文提出了一种无帧内失真漂移的HEVC视频自适应隐写算法。首先,根据帧内预测方向和多系数修改的方法选择合适的载体,进而避免修改系数引起的失真漂移。然后,设计一种包含块内失真和修改载体后的系数分布的失真函数,指导校验网格编码(syndrome trellis codes, STC)对嵌入失真小的载体进行修改。最后,根据最小化失真代价将消息嵌入到满足特定条件的4×4亮度块的量化正弦变换(quantized discrete sinc transform, QDST)系数中。实验结果表明,所提出的视频隐写算法能够有效地避免帧内失真漂移,在获得良好的视觉感知质量的同时,能保证隐写算法的安全性。     

Rate Control Algorithms for Non-Embedded Wavelet-Based Image Coding

During the last decade, there has been an increasing interest in the design of very fast wavelet image encoders focused on specific applications like interactive real-time image and video systems, running on power-constrained devices such as digital cameras, mobile phones where coding delay and/or available computing resources (working memory and power processing) are critical for proper operation. In order to reduce complexity, most of these fast wavelet image encoders are non-(SNR)-embedded and as a consequence, precise rate control is not supported. In this work, we propose some simple rate control algorithms for these kind of encoders and we analyze their impact to determine if, despite their inclusion, the global encoder is still competitive with respect to popular embedded encoders like SPIHT and JPEG2000. In this study we focus on the non-embedded LTW encoder, showing that the increase in complexity due to the rate control algorithm inclusion, maintains LTW competitive with respect to SPIHT and JPEG2000 in terms of R/D performance, coding delay and memory consumption.     

Compression efficiency and delay tradeoffs for hierarchical B-pictures and pulsed-quality frames.

Real-time video applications require tight bounds on end-to-end delay. Hierarchical bidirectional prediction requires buffering frames in the encoder input buffer, thereby contributing to encoder input delay. Long-term frame prediction with pulsed quality requires buffering at the encoder output, increasing the output buffer delay. Both hierarchical B-pictures and pulsed-quality coders involve uneven bit-rate allocation. Both the encoder and decoder buffering requirements depend on the rate allocation. We derive an efficient rate allocation for hierarchical B-pictures using the power spectral density of a wide-sense stationary process. In addition, we discuss important aspects of hierarchical predictive coding, such as the effect of the temporal prediction distance and delay tradeoffs for prediction branch truncation. Finally, we investigate experimentally the tradeoff between delay and compression efficiency.     

Bit-Rate Efficiency of H.264 Encoders Measured With Subjective Assessment Techniques

In this study we evaluated the bit-rate efficiency of current hardware H.264 encoders as compared to that of established MPEG-2 hardware encoders. To estimate bit-rate efficiency, we measured the subjective video quality of MPEG-2 encoded material processed at three bit rates: 8, 12, and 16 Mbps, and determined the bit-rate at which H.264 encoded material produced similar subjective video quality. The MPEG-2 and H.264 bit rates that resulted in the same perceived video quality were used to estimate bit-rate efficiency.      

Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model

Joint video/depth rate allocation is an important optimization problem in 3D video coding. To address this problem, this paper proposes a distortion model to evaluate the synthesized view without access to the captured original view. The proposed distortion model is an additive model that accounts for the video-coding-induced distortion and the depth-quantization-induced distortion, as well as the inherent geometry distortion. Depth-quantization-induced distortion not only considers the warping error distortion, which is described by a piecewise linear model with the video power spectral property, but also takes into account the warping error correlation distortion between two sources reference views. Geometry distortion is approximated from that of the adjacent view synthesis. Based on the proposed distortion model, a joint rate allocation method is proposed to seek the optimal trade-off between video bit-rate and depth bit-rate for maximizing the view synthesis quality. Experimental results show that the proposed distortion model is capable of approximately estimating the actual distortion for the synthesized view, and that the proposed rate allocation method can almost achieve the identical rate allocation performance as the full-search method at less computational cost. Moreover, the proposed rate allocation method consumes less computational cost than the hierarchical-search method at high bit-rates while providing almost the equivalent rate allocation performance.     

Audio and video bit-rate reduction

We address the problem of estimating limits of bit-rate reduction for audio and video information. Binary encoding of such signals is performed by a myriad of different techniques which we classify into two categories. The goal of "waveform" encoders is simply replication of waveforms, whereas "parameter" encoders attempt to extract and transmit only the basic features necessary for the specific application. Encoders are compared in terms of bits per Nyquist sample (η). Minimizing η requires utilization of source signal statistics and human perception. Accordingly, we discuss audio and video sources and their statistics, the salient properties of hearing and vision, and subjective assessment of quality. A review of the present state of waveform and parameter encoding is presented, summarized by graphs of η versus coder complexity for different qualities of reproduction. Current trends are then extrapolated, new directions anticipated and limiting values of η estimated. In summary, we predict that waveform encoders will be refined to the point where bit rates are nearly optimum, but highly variable. Dramatic reductions will only come about through sophisticated parameter encoding.     

An analysis of the efficiency of different SNR-scalable strategies for video coders.

In this paper, we analyze the efficiency of three signal-to-noise scalable strategies for video coders using single-loop motion-compensated prediction (MCP). In our analysis, we assume the video sequences have uniform and constant translational motion and we model MCP as a stochastic filter. We also assume an exponential model for the distortion-rate function of the intraframe coding. The analysis is divided into two parts: the steady-state analysis and the transient analysis. In the first part, only the steady-state response of the coders is taken into account, and, thus, this analysis allows us to asses approximately the efficiency of coders with long input sequences. The transitory analysis considers both the transient and the steady-state responses of the coders, which makes it appropriate to analyze coders using periodic intraframes or with short input sequences. To validate our analysis, theoretical results have been compared to results from encodings of real video sequences using the scalable adaptive motion compensated wavelet video coder. We show that our theoretical analysis effectively describes qualitatively the main trends of every video coding strategy.     

Drift controlled scalable wavelet based video coding in the overcomplete discrete wavelet transform domain

Traditional video coders use the previous frame to perform motion estimation and compensation. Though they are less complex and have minimum coding delays, these coders lose their efficiency when subjected to scalability requirements. Recent 3D wavelet coders using lifting schemes offer high compression efficiency and scalability without significant loss in performance. The main drawback of 3D coders is that they process several frames at a time. This introduces additional delay, which makes them less suitable for real time applications.In this work, we propose a novel scheme to minimize drift in scalable wavelet based video coding, which gives a balanced performance between compression efficiency and reconstructed quality with less drift. Our drift control mechanism maintains two frame buffers in the encoder and decoder; one that is based on the base layer and one that is based on the base plus enhancement layers. Drift control is achieved by switching between these two buffers for motion estimation and compensation. Our prediction is initially based on the base plus enhancement layers buffer, which inherently introduces drift in the system if a part of the enhancement layer is not available at the receiver. A measure of drift is computed based on the channel information and a threshold is set. When the measure exceeds the threshold, i.e., when drift becomes significant, we switch the prediction to be based on the base layer buffer, which is always available to the receiver. We also developed an adaptive scheme with additional computation overhead at the encoder to decide the switching instance. The performance of the threshold case that needs fewer computations is comparable with the adaptive scheme. Our coder offers high compression efficiency and sustained video quality for variable bit rate wireless channels. This proves that we need not completely eliminate drift and decrease compression efficiency to get better received video quality.     

Highly scalable wavelet-based video codec for very low bit-rateenvironment

We introduce a highly scalable video compression system for very low bit-rate videoconferencing and telephony applications around 10-30 kbits/s. The video codec first performs a motion-compensated three-dimensional (3-D) wavelet (packet) decomposition of a group of video frames, and then encodes the important wavelet coefficients using a new data structure called tri-zerotrees (TRI-ZTR). Together, the proposed video coding framework forms an extension of the original zero tree idea of Shapiro (1992) for still image compression. In addition, we also incorporate a high degree of video scalability into the codec by combining the layered/progressive coding strategy with the concept of embedded resolution block coding. With scalable algorithms, only one original compressed video bit stream is generated. Different subsets of the bit stream can then be selected at the decoder to support a multitude of display specifications such as bit rate, quality level, spatial resolution, frame rate, decoding hardware complexity, and end-to-end coding delay. The proposed video codec also allows precise bit rate control at both the encoder and decoder, and this can be achieved independently of the other video scaling parameters. Such a scheme is very useful for both constant and variable bit rate transmission over mobile communication channels, as well as video distribution over heterogeneous multicast networks. Finally, our simulations demonstrated comparable objective and subjective performance when compared to the ITU-T H.263 video coding standard, while providing both multirate and multiresolution video scalability     

DVFS-aware motion estimation design scheme based on bandwidth–rate–distortion optimization in application processor systems

In this paper, a novel dynamic voltage–frequency scaling-aware (DVFS-aware) bandwidth- efficient motion estimation (ME) scheme is presented for mobile application processor (AP) systems. Under volatile operating performance conditions due to the power management mechanism, we model the coding bandwidth (BW) and coding performance for the video processor as a convex function of the working frequency. In this paper, we present a bandwidth–rate–distortion (B–R–D) optimized framework that will guarantee the smallest possible rate–distortion cost among coding BW constraints applied in video coding design. By formulating the coding bandwidth-constrained ME problem as an optimization problem, known convex optimization theory can be applied to yield optimal resource-constrained compression. Using varied CIF (352×288)- and HP (1280×720)-sized video sequences with different motion activities over our proposed DVFS-aware video coding approach, the excellent results in terms of coding performance and coding bandwidth savings are obtained. With negligible quality loss, the proposed scheme under coding BW constraints achieves 45–65% coding BW usage reduction over HD-sized 30 frame/s video coding.     

Scalable video compression via overcomplete motion compensated wavelet coding

Recently, there have been a flurry of works on overcomplete motion compensated wavelet coding (OMCWC). In this paper, we address the importance of phase and focus on the design of scalable video coding algorithms within the OMCWC framework. Specifically, our new contributions consist of the following three components: (1) efficient block motion estimation techniques in the wavelet domain including hierarchical and fractional-pel block matching, (2) extend overcomplete motion compensated prediction (MCP) into overcomplete motion compensated temporal filtering (MCTF) to achieve temporal scalability (3) context modeling strategies for embedded quantization and entropy coding of 3D wavelet coefficients. Experiment results are used to demonstrate that the class of overomplete MCP/MCTF coders are capable of achieving comparable performance to other competing interframe wavelet coders.     

Unequal error protection for MPEG-2 video transmission over wireless channels

This paper proposes an unequal error protection (UEP) method for MPEG-2 video transmission. Since the source and channel coders are normally concatenated, if the channel is noisy, more bits are allocated to channel coding and fewer to source coding. The situation is reversed when the channel conditions are more benign. Most of the joint source channel coding (JSCC) methods assume that the video source is subband coded, the bit error sensitivity of the source code can be modeled, and the bit allocations for different subband channels will be calculated. The UEP applied to different subbands is the rate compatible punctured convolution channel coder. However, the MPEG-2 coding is not a subband coding, the bit error sensitivity function for the coded video can no longer be applied. Here, we develop a different method to find the rate-distortion functions for JSCC of the MPEG-2 video. In the experiments, we show that the end-to-end distortion of our UEP method is smaller than the equal error protection method for the same total bit-rate.     

Implementation of SDTV Video Encoder with Real-Time Constant and Variable Bit-Rate Control

The digital standard definition television (SDTV) encoder is a very important part of the digital TV broadcast chain. Most real-time MPEG-2 encoders are designed to perform in a constant bit-rate (CBR) mode. But an even better compressed stream can be created by employing a variable bit-rate (VBR) encoding algorithm. VBR can be exploited as a means of achieving statistical multiplexing for digital broadcast satellites. This paper suggests an implementation procedure of an SDTV video encoder and proposes a novel VBR bit-allocation strategy that could be implemented in this encoder system. First, using a rate-quantization model and rate-quantization perceptual model, a real-time VBR bit-allocation strategy is deduced. In this strategy, more (or fewer) bits are allocated to "difficult-to-encode" (or "easy-to-encode") groups of pictures (GOPs), which are distinguished according to the estimated encoding complexity of the GOPs. After allocating an appropriate number of bits to each GOP by using this VBR bit-allocation strategy, we use a CBR rate control algorithm to allocate a number of bits and select a quantization scaler for each picture of a GOP. Then smooth visual quality is achieved not only in a GOP but also in the whole video sequence. Second, the system implementation of an SDTV video encoder including a video input module, a video encoding module, a system control and rate control module (SCRCM), and a PES packetizing module is described. We also discuss in detail how to implement our real-time VBR bit-allocation strategy in the SCRCM. Finally, experimental results demonstrate that our proposed VBR encoder displays a better performance than the CBR encoder.     

Scalable image and video transmission using irregular repeat-accumulate codes with fast algorithm for optimal unequal error protection

This paper considers designing and applying punctured irregular repeat-accumulate (IRA) codes for scalable image and video transmission over binary symmetric channels. IRA codes of different rates are obtained by puncturing the parity bits of a mother IRA code, which uses a systematic encoder. One of the main ideas presented here is the design of the mother code such that the entire set of higher rate codes obtained by puncturing are good. To find a good unequal error protection for embedded bit streams, we employ the fast joint source-channel coding algorithm in Hamzaoui et al. to minimize the expected end-to-end distortion. We test with two scalable image coders (SPIHT and JPEG-2000) and two scalable video coders (3-D SPIHT and H.26L-based PFGS). Simulations show better results with IRA codes than those reported in Banister et al. with JPEG-2000 and turbo codes. The IRA codes proposed here also have lower decoding complexity than the turbo codes used by Banister et al.     

Exploiting quantization and spatial correlation in virtual-noise modeling for distributed video coding

Aiming for low-complexity encoding, video coders based on Wyner–Ziv theory are still unsuccessfully trying to match the performance of predictive video coders. One of the most important factors concerning the coding performance of distributed coders is modeling and estimating the correlation between the original video signal and its temporal prediction generated at the decoder.One of the problems of the state-of-the-art correlation estimators is that their performance is not consistent across a wide range of video content and different coding settings. To address this problem we have developed a correlation model able to adapt to changes in the content and the coding parameters by exploiting the spatial correlation of the video signal and the quantization distortion.In this paper we describe our model and present experiments showing that our model provides average bit rate gains of up to 12% and average PSNR gains of up to 0.5 dB when compared to the state-of-the-art models. The experiments suggest that the performance of distributed coders can be significantly improved by taking video content and coding parameters into account.