基于SSIM的HEVC帧内编码率失真优化

率失真优化(RDO)是视频编码压缩中的关键技术之 一。传统的率失真优 化技术使用误差平方和(SSE)或绝对差和(SAD)度量失真, 不能获得较好的视觉感知质量。本文针对新一代视频编码标准高效视频编码(H EVC)帧内编码RDO,提出了一种基于结构相似度(SSIM)度量失真的 RDO方法。首先,根据HEVC编码树形单元(CTU)结构设计了基于SSIM的失真计算方法; 然后,提出了一种基于量化器推导的码率-量化步长(R-Δ)模型和一种基于统计分析的失 真 -量化参数(DSSIM-QP)模型,用于求取拉格朗日乘数;最终,使用多QP优化方法求 取模 型参数。实验表明,相对于HEVC 传统的RDO,针对全I帧(即所有帧都为帧内编码)固定QP编码和多QP优化编 码,在相同的SSIM条 件下,码率分别平均降低8.4%和13.9%左右,同时编码复杂度分别平均增加约3%和2%。     

Content-adaptive deblocking for high efficiency video coding

In this paper, a content-adaptive deblocking method is proposed to improve the visual quality of block-based DCT compressed videos. We find that the edge information obtained through the global orientation energy edge detection (OEED) on an initially deblocked image provides a robust partition of local directional features (LDFs). Based on this partition, for the directional featured region, we design corresponding filter orientation and thresholds to best preserve image details while reducing blocking artifacts; for the consecutive non-featured region, we impose extra smoothing to suppress the visually severe blocking artifacts. Experimental results demonstrate that our method effectively improves the visual quality while well maintaining the objective fidelity of block-based DCT compressed videos, compared with the H.264/AVC deblocking method and other existing directional deblocking methods.     

Adaptive pre-interpolation filter for high efficiency video coding

The proposed interpolation filter comprises two concatenating filters, adaptive pre-interpolation filter (APIF) and the normative interpolation filter in H.264/AVC. The former is applied only to the integer pixels in the reference frames; the latter generates all the sub-position samples, supported by the output of APIF. The convolution of APIF and the standard filter minimizes the motion prediction error on a frame basis. APIF preserves the merits of the adaptive interpolation filter (AIF) and the adaptive loop filter (ALF) in the key technical area (KTA) software and at the same time overcomes their drawbacks. The experimental results show that APIF outperforms either AIF or ALF. Compared with the joint use of AIF and ALF, APIF provides comparable performance, but has much lower complexity.     

Complexity-based intra frame rate control by jointing inter-frame correlation for high efficiency video coding

Rate control is of great significance for the High Efficiency Video Coding (HEVC). Due to the high efficiency and low complexity, the R-lambda model has been applied to the HEVC as the default rate control algorithm. However, the video content complexity, which can help improve the code efficiency and rate control performance, is not fully considered in the R-lambda model. To address this problem, an intra-frame rate control algorithm, which aims to provide improved and smooth video quality, is developed in this paper by jointly taking into consideration the frame-level content complexity between the encoded intra frames and the encoded inter frame, as well as the CTU-level complexity among different CTUs in texture–different regions for intra-frame. Firstly, in order to improve the rate control efficiency, this paper introduces a new prediction measure of content complexity for CTUs of intra-frame by jointly considering the inter-frame correlations between encoding intra frame and previous encoded inter frames as well as correlations between encoding intra frame and previous encoded intra frame. Secondly, a frame-level complexity-based bit-allocation-balancing method, by jointly considering the inter-frame correlation between intra frame and previous encoded inter frame, is brought up so that the smoothness of the visual quality can be improved between adjacent inter- and intra-frames. Thirdly, a new region-division and complexity-based CTU-level bit allocation method is developed to improve the objective quality and to reduce PSNR fluctuation among CTUs in intra-frame. In the end, related model parameters are updated during the encoding process to increase rate control accuracy. As a result, as can be seen from the extensive experimental results that compared with the state-of-the-art schemes, the video quality can be significantly improved. More specifically, up to 10.5% and on average 5.2% BD-Rate reduction was achieved compared to HM16.0 and up to 2.7% and an average of 2.0% BD-Rate reduction was achieved compared to state-of-the-art algorithm. Besides, a superior performance in enhancing the smoothness of quality can be achieved, which outperforms the state-of-the-art algorithms in term of flicker measurement, frame and CTU-wise PSNR, as well as buffer fullness.     

Perceptual quality metric for digital video coding

A new perceptually significant block-edge impairment metric as a quantitative distortion measure for blocking artifacts in digital video coding is presented. This distortion metric does not require the original image sequence as a comparative reference, and is found to be consistent with subjective evaluation.     

面向HEVC的恰可察觉编码失真模型

为进一步提高现有视频编码技术的压缩效率及解 码重建图像的主观视觉感知质量,在现有人眼恰可 察觉失真(JND,just noticeable distortion)模型的基础上, 提出了恰可察觉编码失真(JNCD,just noticeable coding dist ortion)模型。首先,通过主观实验,对恰可察觉梯 度幅值差异(JNGD,just noticeable gradient difference)进 行了研究,分析其变化规律并建立JNGD模型。使用全变 分(TV,total variation)方法将图像分解为结构图和纹理图后,分别求 取其梯度信息得到结构梯度图和纹理梯度图, 利用JNGD模型分别滤除结构梯度图和纹理梯度图中的人眼不可察觉的梯度幅值 ;其后,分析了人眼感知对于不同 梯度幅值的编码失真敏感性,设计了梯度幅值与JNCD值的主观实验,得到两者的关系模型; 最后,考虑人眼对图 像中的边缘、平坦和纹理3类区域失真感知程度的差异性,利用滤波后的结构梯度和纹理梯 度信息将图像划分为上 述3类区域,最终建立整幅图像的JNCD模型。为验证本文提出的JNCD模型的可靠性,在高效 视频编码(HEVC)标准测试平台上进行的模型验证结果表明,在本模 型指导下的编码其解码重建 图像获得了较好的主观视觉效果,可为人眼视觉感知冗余的分析及感知编码的改 进提供依据。     

Spatial quality index based rate perceptual-distortion optimization for video coding

Spatial Quality Index (SQI) is a recently proposed video quality assessment metric that can predict video quality much close to subjective judgments. Since current video coding still has much redundancy in the sense of visual perception, in this paper, we incorporate SQI into video coding to further improve compression ratio without visual quality loss. Firstly, contributions of different human visual system (HVS) properties used in SQI are analyzed. Then two most important HVS properties, i.e. contrast masking effect (CME) and motion masking effect (MME), are extracted to measure perceptual-distortion D_p in video coding. Finally, based on D_p, a rate perceptual-distortion optimization (RpDO) algorithm is presented by adopting a suitable Lagrange multiplier from previous study. Experimental results show that, RpDO can averagely achieve 14% bitrate reduction when compared to HM14.0 under the same visual quality. At the same time, there is no significant change in the encoding time.     

基于率失真优化图像编码的研究

数字图像处理技术的一个非常重要的环节是数字图像压缩技术.其在图像保存和图像传输领域拥有无可替代的作用.矢量量化方法是图像压缩技术中一个重要的组成部分.通过有效地转换图像中像素点,从而高效地对图像进行压缩,为图像的存储和传输提供了可靠的保证.本文详细研究了率失真模型在图像压缩领域的应用.通过高效地结合量化失真和编码码率,从而得到在码率一定的条件,相对更高质量的图像.通过对码率分配策略的研究,根据码率和失真二者的关系推导出矢量量化小波图像编码器的率失真模型,同时将该模型与拉格朗日极值算法有机地结合在一起,从而得到图像不同子带的最优码率分配.本文实验结果显示,根据图像的不同子带的特性,应用本文所提出的的图像率失真模型,可以得到较高质量的还原图像.因此可以得出,通过应用本文的率失真模型,在还原图像的还原质量不下降的条件下,得到更低的图像压缩码率.     

Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model

Joint video/depth rate allocation is an important optimization problem in 3D video coding. To address this problem, this paper proposes a distortion model to evaluate the synthesized view without access to the captured original view. The proposed distortion model is an additive model that accounts for the video-coding-induced distortion and the depth-quantization-induced distortion, as well as the inherent geometry distortion. Depth-quantization-induced distortion not only considers the warping error distortion, which is described by a piecewise linear model with the video power spectral property, but also takes into account the warping error correlation distortion between two sources reference views. Geometry distortion is approximated from that of the adjacent view synthesis. Based on the proposed distortion model, a joint rate allocation method is proposed to seek the optimal trade-off between video bit-rate and depth bit-rate for maximizing the view synthesis quality. Experimental results show that the proposed distortion model is capable of approximately estimating the actual distortion for the synthesized view, and that the proposed rate allocation method can almost achieve the identical rate allocation performance as the full-search method at less computational cost. Moreover, the proposed rate allocation method consumes less computational cost than the hierarchical-search method at high bit-rates while providing almost the equivalent rate allocation performance.     

Mode dependent down-sampling and interpolation scheme for high efficiency video coding

In this paper, a mode dependent down-sampling and interpolation scheme is proposed to improve the coding efficiency of the intra prediction module. In the proposed method, we elaborately design the down-sampling structures and interpolation schemes for each directional intra prediction mode by minimizing the spatial prediction distance. The sampled pixels are predicted with a traditional directional intra prediction scheme, and the non-sampled pixels are predicted from the interpolation of their neighboring reconstructed sampling pixels. Both the residuals of the sampled and non-sampled pixels are encoded at last. Experimental results show that the proposed method achieves an average 7.52% bitrate reduction relative to KTA reference software. Since the down-sampling structure and interpolation method is only related to the intra mode, there is no additional overhead at the encoder.     

Adaptive bit allocation scheme for rate control in high efficiency video coding with initial quantization parameter determination

One of the challenges in rate control (RC) lies in how to efficiently determine a target bit rate that will be used for the quantization parameter (Q_p) calculation process during video coding. In this paper, we investigate the issues over the existing bit allocation algorithms for the RC process in High Efficiency Video Coding (HEVC) and propose an complexity-based bit allocation scheme to improve the encoding performance. First, we model the relationship between encoding bit rate and texture complexity by a linear rate function. Second, compared with traditional complexity estimation methods, a more accurate model is proposed to measure the texture complexity considering the spatial–temporal correlations. Third, based on the proposed rate function and texture complexity measurement model, we develop an adaptive bit allocation scheme for RC in HEVC. At the same time, depending on the encoder buffer status, an adaptive Q_p clip range determination algorithm is also developed to achieve the encoding quality smoothness while keeping the bit rate fluctuation at an acceptable level. Then, we exploit to determine the initial Q_p efficiently and adaptively according to video contents. Experimental results demonstrate that the proposed RC algorithm can achieve better rate-distortion (R–D) and rate-control performance than that of the state-of-the-art RC scheme implemented in the HEVC reference software HM11.0.     

Towards high efficiency video coding: Subjective evaluation of potential coding technologies

This paper describes the details and the results of the subjective quality evaluation performed at EPFL, as a contribution to the effort of the joint collaborative team on video coding (JCT-VC) for the definition of the high efficiency video coding (HEVC) standard. The performance of twenty-seven coding technologies has been evaluated with respect to two H.264/MPEG-4 AVC anchors, for high definition (HD) test material. The test campaign involved a total of 494 naive observers and took place over a period of four weeks. While similar tests have been conducted as part of the standardization process of previous video coding technologies, the test campaign described in this paper is by far the most extensive in the history of video coding standardization. A detailed statistical analysis of the subjective results is provided. The results show high consistency and support an accurate comparison of the performance of the different coding technologies.     

基于零块分析的高效视频编码通信

为了实现高清、超高清视频实时编码通信传输, 针对高效视频编码(HEVC)帧间编码计算复杂度过高的问题,根据图像的文理复杂度和 编码单元的零块统计特征,提出一种新的HEVC快速帧间模式判决算法。根据Merge模式下 整单元一分为四的4个子编码单元纹 理相似度确定是否提前终止编码单元(CU)划分,同时利用帧间2N×2N预测模式下零系数与非零系数分布的区域统计特征,选择符合零块分 布特征的最佳预测单元(PU)模式。实验结果表明,在低延迟B(LDB,low-delay B)和随机访 问(RA,random access)配置条件下,提出的算法在保持编码 性能基本不变的情况下,HEVC帧间预测编码时间分别平均减少了60.2%与59.4%。     

Perceptual stereoscopic video coding using disparity just-noticeable-distortion model

In this paper, we propose perceptual stereoscopic video coding using a disparity just-noticeable-distortion (JND) model. We obtain the disparity JND model in stereo videos by disparity masking effects of the human visual system (HVS). The disparity JND model represents the maximum distortion of stereo perception that HVS cannot perceive. Based on the disparity JND model, we adjust prediction residuals to remove the perceptual redundancy of stereo videos. Thus, we achieve significant bit-rate saving while maintaining visual quality. Experimental results demonstrate that the proposed method significantly improves coding efficiency without loss of stereoscopic perceptual quality.     

Gamma rate theory for causal rate control in source coding and video coding

The rate distortion function in information theory provides performance bounds for lossy source coding. However, it is not clear how to causally encode a Gaussian sequence under rate constraints while achieving R–D optimality. This problem has significant implications in the design of rate control for video communication. To address this problem, we take distortion fluctuation into account and develop a new theory, called gamma rate theory, to quantify the trade-off between rate and distortion fluctuation. The gamma rate theory implies that, to evaluate the performance of causal rate controls in source coding, the traditional R–D metric needs to be replaced by a new GRD metric. The gamma rate theory identifies the trade-off between quality fluctuation and bandwidth, which is not known previously. To validate the gamma rate theory, we design a rate control algorithm for video coding; our experimental results demonstrate the utility of the gamma rate theory in video coding.     

Inter-layer correlation-based adaptive bit allocation for enhancement layer in scalable high efficiency video coding

To facilitate the diversity of network and end-user devices, bit allocation technology must be combined with scalable high efficiency video coding (SHVC) to achieve continuous bitrate variation. However, to date, the multilayer coding feature in SHVC has not been fully utilized. In this paper, a bit allocation algorithm is proposed for the enhancement layer in SHVC. The algorithm includes two parts: a bit allocation method for the initial frame and another for subsequent frames. The bit allocation method for the initial frame consists of a bit allocation factor model constructed based on investigation of the influence of the initial target bitrate on the overall coding performance. For the bit allocation of the subsequent frames, alternate GOP coding is designed and implemented to fully exploit the inter layer correlation. In addition, an adaptive frame layer bit allocation ratio model is deduced according to the rate distortion optimization theory. Experimental results show that the proposed algorithm can improve PSNR by 0.44 dB and 0.41 dB under low delay and random-access configurations, respectively, and achieve high bitrate control accuracy. The algorithm also outperforms the state-of-the-art algorithms in term of PSNR improvement.     

A layered coding system for very low bit rate video coding

Known coding techniques for transmitting moving images at very low bit rates are explained by the source models on which these coding techniques are based. It is shown that with motion-compensated hybrid coding, object-based analysis-synthesis coding, knowledge-based coding and semantic coding, there is a consistent development of source models. In consequence these coding techniques can be combined in a layered coding system. From experimental results obtained for object-based analysis-synthesis, coding estimates for the coding efficiency of such a layered coding system are derived using head and shoulder video telephone test sequences. It is shown that an additional compression factor of about 3 can be expected with such a complex layered coding system, when compared to block-based hybrid coding.     

MPEG coding for variable bit rate video transmission

For real-time transmission of broadcast-quality video on ATM-based B-ISDN, the intraframe to interframe ratio and the quantizer scale are two key parameters that can be used to control a video source in a network environment. Their impact on the traffic characteristics of the coder provides insights into the cell arrival process for an MPEG source. The authors obtain video source models for coders that utilize a standard algorithm that can be applied to a multitude of video services. The output stream of a video coder, which complies with the Motion Pictures Expert Group (MPEG) coding standard, is studied with an National Television Systems Committee (NTSC) quality video sequence as the input. Because the MPEG video coding algorithm has been proposed for a variety of applications, they also investigate the effect of changing the coding parameters on the statistics of interest     

Wyner-Ziv视频编码中无反馈速率控制算法研究

为了避免在分布式视频编码系统中使用反馈信道,提出了一种基于Wyner-Ziv编码的无反馈速率控制算法。首先,利用目标码率和目标帧率进行GOP层码率分配;然后,根据原始图像的帧间相关性动态选择量化因子和量化矩阵来分配每个GOP内关键帧和Wyner-Ziv帧的比特数;接下来,利用系数带级的相关性计算相关噪声模型参数,并选择对应的LDPC校验矩阵,提出Wyner-Ziv帧的无反馈比特面速率控制算法。实验结果表明,在给定目标码率下,所提算法的编码码率误差小于0.57%,且与现有无反馈速率控制算法相比,解码恢复图像的PSNR(峰值信噪比)可以提高1dB。另外,该算法基本没有增加编码端复杂度,可用于实际分布式视频通信系统。