Adaptive bit allocation scheme for rate control in high efficiency video coding with initial quantization parameter determination

One of the challenges in rate control (RC) lies in how to efficiently determine a target bit rate that will be used for the quantization parameter (Q_p) calculation process during video coding. In this paper, we investigate the issues over the existing bit allocation algorithms for the RC process in High Efficiency Video Coding (HEVC) and propose an complexity-based bit allocation scheme to improve the encoding performance. First, we model the relationship between encoding bit rate and texture complexity by a linear rate function. Second, compared with traditional complexity estimation methods, a more accurate model is proposed to measure the texture complexity considering the spatial–temporal correlations. Third, based on the proposed rate function and texture complexity measurement model, we develop an adaptive bit allocation scheme for RC in HEVC. At the same time, depending on the encoder buffer status, an adaptive Q_p clip range determination algorithm is also developed to achieve the encoding quality smoothness while keeping the bit rate fluctuation at an acceptable level. Then, we exploit to determine the initial Q_p efficiently and adaptively according to video contents. Experimental results demonstrate that the proposed RC algorithm can achieve better rate-distortion (R–D) and rate-control performance than that of the state-of-the-art RC scheme implemented in the HEVC reference software HM11.0.     

Window-based rate control for video quality optimization with a novel INTER-dependent rate-distortion model

Most model-based rate control schemes use independent rate-distortion (R–D) models at macroblock (MB) level to represent the relationship among bit rate, distortion and encoding complexity. However the correlations between frames (INTER-dependency) are not well considered for distortion, bit allocation and quantization parameter (QP) decision. In this paper, a novel INTER-dependent R–D model is proposed based on the theoretical analysis of the relationship between the predicted residual of one frame and the distortion of its reference frame. To achieve both bit rate accuracy and consistent video quality, a window-based rate control scheme with two sliding windows is introduced. One window is to group certain previously encoded frames and current frame to control the bit rate and buffer delay; the other is to group certain future encoding frames to optimize the fluctuation of video quality. Furthermore, the optimization of Lagrange multiplier is also discussed under the INTER-dependent situation. Experimental results demonstrate that the proposed window-based rate control scheme with INTER-dependent R–D model can achieve accurate target bit rate and improve PSNR performance, meanwhile the variation of PSNR is the smallest compared with other three benchmark algorithms. This one-pass rate control scheme is highly practical for the real-time video coding applications.     

New rate-distortion modeling and efficient rate control for H.264/AVC video coding

Rate control (RC) is crucial in controlling compression bit rates and qualities for networked video applications. In this paper, we propose a new rate-distortion (R-D) model and an efficient rate control scheme for H.264/AVC video coding, which elegantly resolve the inter-dependency problem between rate-distortion optimization and rate control by eliminating the need of coding complexity prediction for an inter-frame. The objective is to achieve accurate bit rate, obtain optimal video quality while reducing quality variations and simultaneously handling buffer fullness effectively. The proposed algorithm encapsulates a number of new features, including a coding complexity measure for intra-frames, a rate-distortion model, an accurate quantization parameter (QP) estimation for intra-frames, an incremental quantization parameter calculation method for inter-frames, a proportional+integral+derivative (PID) buffer controller, and an intelligent bit-allocation-balancing technique. Our experimental results demonstrate that the proposed scheme outperforms the JVT-G012 solution by providing accurate rate regulation, effectively reducing frame skipping, and finally improving coding quality by up to 1.80 dB.     

基于Weibull率失真简化模型的帧比特分配算法

为在确保视频压缩码流平稳且带宽受限的情况下提升重建视频质量,本文针对使用Weibull密度分布对压缩视频序列率失真性能分析的方法,对其模型进行数学简化,并以帧层编码速率控制的缓冲器充盈度和重建视频质量为目标,提出了一种新的H.264压缩视频速率控制方法——帧比特分配算法。该算法以H.264的JM16.1为视频压缩编码平台,将简化率失真模型嵌入速率控制算法中,以一个GOP为控制单元对其内部各图像帧进行比特分配与调整,同时以帧为周期进行算法模型参数更新。实验结果表明,本文提出的算法要比JVT-H017的算法在重建视频客观质量（PSNR）方面平均有0.30dB的改善,并能够得到更加平稳质量的视频输出。在实际输出码率方面,本文算法比JVT-H017的算法更低,并且编码缓冲器充盈度较JVT-H017平稳且没有上下溢的情况出现。      

Spatial quality index based rate perceptual-distortion optimization for video coding

Spatial Quality Index (SQI) is a recently proposed video quality assessment metric that can predict video quality much close to subjective judgments. Since current video coding still has much redundancy in the sense of visual perception, in this paper, we incorporate SQI into video coding to further improve compression ratio without visual quality loss. Firstly, contributions of different human visual system (HVS) properties used in SQI are analyzed. Then two most important HVS properties, i.e. contrast masking effect (CME) and motion masking effect (MME), are extracted to measure perceptual-distortion D_p in video coding. Finally, based on D_p, a rate perceptual-distortion optimization (RpDO) algorithm is presented by adopting a suitable Lagrange multiplier from previous study. Experimental results show that, RpDO can averagely achieve 14% bitrate reduction when compared to HM14.0 under the same visual quality. At the same time, there is no significant change in the encoding time.     

Novel rate control scheme for intra frame video coding with exponential rate–distortion model on H.264/AVC

Rate control regulates the output bit rate of a video encoder in order to obtain optimum visual quality within the available network bandwidth and to maintain buffer fullness within a specified tolerance range. Due to the benefits of intra-only encoding, such as less computational cost and less latency, it has been more and more widely used. In this paper, we propose an accurate intra-only rate control scheme for H.264/AVC, which includes a novel complexity measurement and a new rate–distortion (R–D) model. We also propose a linear rate–complexity model which takes the intercept into consideration to reduce the estimation error. The proposed R–D model is integrated by the linear rate–complexity model and an exponential rate–quantization model. Based on theoretical analysis and experimental validation, the proposed scheme has high bits prediction precision, and it can also accurately handle buffer fullness. Compared with JVT-W042, our algorithm achieves higher average PSNR and improves the coding quality up to 0.35 dB.     

Optimized video compression with residual split attention and swin-block artifact contraction

Research in video compression has seen significant advancement in the last several years. However, the existing deep learning-based algorithms continue to be plagued by erroneous motion compression and ineffective motion compensation architectures, resulting in compression errors with a lower rate–distortion trade-off. To overcome these challenges, we present an end-to-end purely deep learning-based video compression method through a set of primary operations (e.g., motion estimation, motion compression, motion compensation, residual compression, and artifact contraction) differently. A deep residual attention split (DRAS) block is introduced for motion compression networks to pay more attention to certain image regions to create more effective features for the decoder while boosting the rate–distortion optimization (RDO) efficiency. A channel residual block (CRB) is proposed in motion compensation to yield a more accurate predicted frame, potentially improving the residual frame. To mitigate the compression errors, an artifact contraction module (ACM) by residual swin convolution UNet block is included in this model to improve the reconstruction quality. To improve the final frame, a buffer is added to fine-tune the previous reference frames. These modules combine with a loss function by assessing the trade-off and enhancing the decoded video quality. A comprehensive ablation study demonstrates the effectiveness of the proposed blocks and modules for video compression. Experimental results show the competitive performance of the proposed method on four benchmark datasets.     

A rate-control algorithm using inter-layer information for H.264/SVC for low-delay applications

This paper proposes a novel frame-level rate-control scheme for H.264/SVC for low-delay applications. The trend of the Mean Absolute Difference (MAD) (after motion compensation) of the reference layer is used as the inter-layer information for Inter-layer MAD prediction, which is combined with Intra-layer MAD prediction to predict the MAD of the enhancement layers (EL) more accurately. By considering the influence from both the texture and nontexture information, the varying picture complexity and feedback information from the actual encoding results are combined to achieve an accurate bit-allocation. A coarse-to-fine initial quantization parameter (QP) selection method is proposed to refine the initial QP of the EL according to the channel condition and the video sequence characteristics. Experimental results demonstrate that the proposed scheme can obtain high and smooth PSNR, and the output bit-rate is close to the target bit-rate. Also, the proposed scheme can avoid serious buffer fullness fluctuation and reduce skipped frames in the video coding.     

全I帧编码中的TM5码率控制策略的改进

TM5码率控制策略作为一种非常有效的控制策略而被MPEG-2编码器所广泛采用，但它仍存在诸多的不足。本文针对全I帧编码下的TM5码率控制策略，提出了TM5码率控制策略的改进方案，这包括利用宏块的DCT系数来实现宏块空间活动性的快速计算，以及使用新的缓冲器充溢度调整方法。实验表明：与采用TM5码率控制策略的全I帧图像编码相比，本文提出的改进方案的效果非常显著。     

Improved rate control for MPEG-4 video transport over wireless channel

Video streaming over wireless channel is challenging due to a number of factors such as limited bandwidth and loss sensitivity. In this paper, we develop a novel rate control algorithm for MPEG-4 video coding. Unlike traditional rate control schemes, we jointly consider the encoding complexity variation and buffer variation as well as human visual properties to optimize the rate control efficiency. We also analyze the sensitivity of a macroblock (MB) as a result of bit errors and calculate its error sensitivity metric. This metric is used in unequal error protection of the MB. Simulation results show that proposed approach can improve the decoded picture quality in wireless video coding and transmission.     

基于内容复杂度的视频编码码率控制

该文根据线性模型提出信源内容复杂度的一种度量方式,并以此进行帧级比特分配和宏块级的视频码率控制。仿真表明该方法与MPEG2中的TM5相比,能比较显著地提高信噪比,实现更加准确的码率控制,同时可以在帧间和帧内获得更一致的视觉效果。该算法在MPEG4及低码率视频编码中也是适用的。     

Gamma rate theory for causal rate control in source coding and video coding

The rate distortion function in information theory provides performance bounds for lossy source coding. However, it is not clear how to causally encode a Gaussian sequence under rate constraints while achieving R–D optimality. This problem has significant implications in the design of rate control for video communication. To address this problem, we take distortion fluctuation into account and develop a new theory, called gamma rate theory, to quantify the trade-off between rate and distortion fluctuation. The gamma rate theory implies that, to evaluate the performance of causal rate controls in source coding, the traditional R–D metric needs to be replaced by a new GRD metric. The gamma rate theory identifies the trade-off between quality fluctuation and bandwidth, which is not known previously. To validate the gamma rate theory, we design a rate control algorithm for video coding; our experimental results demonstrate the utility of the gamma rate theory in video coding.     

Adaptive rate control algorithm based on logarithmic R–Q model for MPEG-1 to MPEG-4 transcoding

This paper proposes an adaptive rate control algorithm for MPEG-1 to MPEG-4 transcoder for video internetworking where a compressed video is transmitted through networks of different capacities. It is based on an adaptive logarithmic R–Q (rate-quantization) model for controlling the new target bit-rate of a transcoded MPEG-4 bitstream. In R–Q modeling, the R–Q characteristics predicted from a previous picture of the same picture type are used in combination with the R–Q data measured for the current picture. The proposed adaptive model takes into account the effect of picture dependencies arising in motion-compensated video coding. Furthermore, full advantage is taken of the special situation related to transcoding where a priori knowledge of the current R–Q data can be obtained by just counting the number of bits in the input bitstream without re-encoding. It is shown by simulation that the adaptive R–Q model-based rate control can efficiently control the output bit-rate to match the target bit-budget.     

Rate-constrained arbitrarily shaped video object coding with object-based rate control

The authors develop an object-based rate control algorithm for MPEG-4 arbitrarily shaped video object coding. There are three main contributions. An improved quadratic rate-distortion model for texture information is derived, which is more accurate for arbitrarily shaped video objects. Secondly, an accurate rate-distortion model for shape information is introduced. Combining the rate-distortion models for texture and shape information, an object level rate control is proposed. To achieve constant bit rate, the quantiser is adjusted based on buffer fullness. The tradeoff between picture quality and buffer fullness is considered. Compared with the rate control algorithm for the MPEG-4 verification model, the proposed scheme can achieve a more stable buffer without sacrificing picture quality.     

Nonlinear HEMT model formulated from the second-order derivative of the I-V and Q-V characteristics

In this paper, an empirical nonlinear model of high electron mobility transistors (HEMTs) suitable for a wide bias range is presented. Unlike the conventional large-signal models whose fitting parameters are coupled to the measured drain current and gate capacitance characteristics, the derived modeling equations are direct formulated from the second-order derivative of drain current (I-V) and gate charge (Q-V) with respect to gate voltage. As a consequence, the proposed nonlinear model is kept continuously differentiable and accurate enough to the higher-order I-V and Q-V derivatives. Besides, the thermal and trapping effects have been implemented in the large-signal model along with its dependence on temperature and quiescent-bias state. The composite nonlinear model is shown to accurately predict the S-parameters, large-signal power performances as well as the two-tone intermodulation distortion products for various types of GaAs and GaN HEMTs.     

An analytic study of a scalable video buffer

In this contribution, we investigate the performance of the output buffer of an ‘on-demand’ video streaming server. The server maintains a local database of stored video clips and movies which can be streamed to the users upon request. We assume that the stored video is encoded in a scalable way, which means that the data streams contain a base layer ensuring a minimum of guaranteed quality and a stack of additional enhancement layers progressively improving the quality of the video. For the purpose of performance analysis, we assume that a video stream is split up in logical units called frames. Every frame consists of a number of packets, each containing information of one layer only. When the output buffer gets congested, one may choose to drop the transmission of some of the layers in a frame, thus reducing the frame transmission time and expediting the restoration of the buffer size to normal levels. A discrete-time finite capacity queueing model with buffer size dependent transmission times is proposed. Using a probability generating function approach, we focus on the characteristics of idle and busy periods. We obtain performance measures such as the frame loss ratio and the average frame transmission time. The latter measure relates to the quality of the video stream. We conclude with some numerical examples, including a realistic case study.     

Just noticeable distortion model and its applications in video coding

We explore a new perceptually-adaptive video coding (PVC) scheme for hybrid video compression, in order to achieve better perceptual coding quality and operational efficiency. A new just noticeable distortion (JND) estimator for color video is first devised in the image domain. How to efficiently integrate masking effects together is a key issue of JND modelling. We integrate spatial masking factors with the nonlinear additivity model for masking (NAMM). The JND estimator applies to all color components and accounts for the compound impact of luminance masking, texture masking and temporal masking. Extensive subjective viewing confirms that it is capable of determining a more accurate visibility threshold that is close to the actual JND bound in human eyes. Secondly, the image-domain JND profile is incorporated into hybrid video encoding via the JND-adaptive motion estimation and residue filtering process. The scheme works with any prevalent video coding standards and various motion estimation strategies. To demonstrate the effectiveness of the proposed scheme, it has been implemented in the MPEG-2 TM5 coder and demonstrated to achieve average improvement of over 18% in motion estimation efficiency, 0.6 dB in average peak signal-to perceptual-noise ratio (PSPNR) and most remarkably, 0.17 dB in the objective coding quality measure (PSNR) on average. Theoretical explanation is presented for the improvement on the objective coding quality measure. With the JND-based motion estimation and residue filtering process, hybrid video encoding can be more efficient and the use of bits is optimized for visual quality.     

一种空域可分级的自适应视频码流控制方法

在传统的CBR码流控制中,为防止缓冲器上溢,通常采用跳帧的方法来保持码率恒定。由于跳帧引起解码端运动补偿误差增大,客观上造成跳帧期间图像PSNR降低,图像失真度迅速增加,在主观上造成图像有跳跃感,画面不流畅。针对以上缺陷,该文基于H.263+的TMN8帧级码流控制算法,提出了一种空域可分级的自适应码流控制方法。该方法在编码端对原跳帧方法中不编码的帧进行自适应下采样编码,解码端通过上采样恢复。实验结果证明,该文提出的码流控制算法在低带宽条件下,图像质量较跳帧方法有较大提高。在低码率条件下,对于高运动图像该方法的平均PSNR值比采用跳帧的码流控制方法高0.3~0.8dB;比采用增大量化步长的方法高0.1~0.4dB,而对于跳帧阶段PSNR值,该算法比传统的跳帧方法高1~2dB左右。