A novel fast intra mode decision for versatile video coding

The latest video coding standard Versatile Video Coding (VVC) obtains superior coding efficiency compared to the High Efficiency Video Coding (HEVC), which is achieved by incorporating more effective and complex new coding tools. In this paper, we propose a novel fast intra mode decision algorithm for VVC, including following two strategies: (1) the correlation between the optimal modes of the adjacent blocks and the modes selected in the rough modes decision (RMD) process is analyzed and applied to reduce the modes in the candidate list; (2) modes in the candidate list are sorted in ascending order according to the modes’ cost calculated in the RMD process. An early termination method is proposed for terminating the optimal prediction mode decision process based on this new order early. These two strategies are incorporated into intra coding to reduce the coding complexity. Since these two strategies do not add any additional computational complexity, the proposed fast algorithm can achieve more complexity reduction. The experimental results show that the complexity reduction of the proposed algorithm is up to 44.74% compared to VVC reference software VTM2.0, and averagely 30.59% encoding time saving with 0.86% BDBR increase.     

Efficient prediction of CU depth and PU mode for fast HEVC encoding using statistical analysis

High Efficiency Video Coding (HEVC) adopts complex quadtree-structured CU (coding unit) and PU (prediction unit) modes. Thus, the search process for optimal modes causes high computational complexity in HEVC. To solve this problem, a fast coding algorithm is proposed. Thirteen neighboring CTUs (Coding Tree Unit) are divided into three classes with the k-means method, and are selectively used as the reference CTUs to calculate the predicted CU depth levels for the current CTU. Additionally, for every CU depth, we skip some rarely used PU modes to reduce the complexity of the PU selection algorithm. Experimental results show that compared with the HEVC reference software, our algorithm can achieve 56.71% time saving with Low Delay Configuration profile and 59.76% with Random Access Configuration profile, whereas values of BDBR (Bjontegaard Delta Bit Rate) are only 1.0517% and 0.9918%, respectively. Compared with five state-of-the-art algorithms, the proposed algorithm has significant encoding time reduction with good bitrate performances.     

A novel fast intra algorithm for VVC based on histogram of oriented gradient

The latest Versatile Video Coding (VVC) standard incorporates a series of effective and complex new intra coding tools, which obtains superior coding efficiency than the High Efficiency Video Coding (HEVC). However, this makes the intra coding more complicated and time-consuming. A fast algorithm for VVC from two aspects of fast mode decision and fast partition decision is proposed in this paper. For the fast mode decision, the relationship between bins with Histogram of Oriented Gradient (HOG) and intra modes is created for the mode selection, decreasing the planar modes for SATD and RDO. Moreover, we analyze the maximum bins to determine the final modes, and we use the modes of left and upper blocks as a reference for the current CU, which can early terminate RDO. Moreover, a two-step fast partition algorithm is proposed based on HOG for fast partition decision, in which two thresholds are investigated to control the uniformity of textures. The proposed fast algorithm is implemented on the VVC test model, and the experimental results show that it can achieve 69.07% time savings with only 2.96% BDBR increases averagely, which outperforms other relatively existing state-of-the-art methods. Moreover, to convince the universality of our algorithm, we further implement our method in Fraunhofer Versatile Video Encoder (VVenc) and Fraunhofer Versatile Video Decoder (VVdec), which have five settings to control the trade-off between encoding quality and efficiency for intra coding. The fast intra mode decision algorithm and fast partition algorithm decrease the complexity of intra coding for both VTM and VVenc, which shows the efficiency and universality of the proposed fast partition and fast mode decision algorithms.     

Fast algorithm based on the sole- and multi-depth texture measurements for HEVC intra coding

In High Efficiency Video Coding (HEVC), intra coding plays an important role, but also involves huge computational complexity due to a flexible coding unit (CU) structure and a large number of prediction modes. This paper presents a fast algorithm based on the sole- and multi-depth texture measurements to reduce the complexity from CU size and prediction mode decisions. For the CU size decision, evaluation results in the CU coding with one and multiple depths are utilized to classify CUs into heterogeneous, homogeneous, depth-prominent and other ones. Fast CU size decisions are made for different kinds of CUs. For the prediction mode decision, the tendencies for different CU sizes are detected based on multiple depths. The number of searching modes is decreased adaptively for the CU size with fewer tendencies. Experimental results show the proposed algorithm by off-line training reduces 53.32% computational complexity, with 1.47% bit-rate increasing.     

Low-Complexity Intra-Coding Scheme for HEVC

Compared with H.264/AVC, the latest video standard High Efficiency Video Coding (HEVC) known as H.265 improves the coding efficiency by adopting the quadtree splitting structure which is flexible in representing various textural and structural information in images. However, the computational complexity is dramatically increased, especially in the intra-mode decision process owing to supporting more partitions and modes. In this paper, we propose a low-complexity algorithm for HEVC intra-coding, which consists of a fast coding unit (CU) size decision (FCUSD) method and a fast prediction unit (PU) mode decision (FPUMD) method. In FCUSD, unnecessary CU sizes are skipped early according to the depth level of neighboring CUs and the rate distortion (RD) cost threshold derived from the former coded frame. In FPUMD, the PU mode and RD cost correlations between different depth levels are utilized to terminate unnecessary candidate modes. Experimental results demonstrate that the proposed algorithm can achieve about 50.99 % computational complexity reduction on average with 1.18 % BD-rate increase and 0.08 dB BD-psnr loss.     

基于感兴趣区域的高性能视频编码帧内预测优化算法

针对高性能视频编码(HEVC)帧内预测编码算法复杂度较高的问题,该文提出一种基于感兴趣区域的高性能视频编码帧内预测优化算法。首先,根据图像显著性划分当前帧的感兴趣区域(ROI)和非感兴趣区域(NROI);然后,对ROI基于空域相关性采用提出的快速编码单元(CU)划分算法决定当前编码单元的最终划分深度,跳过不必要的CU划分过程;最后,基于ROI采用提出的预测单元(PU)模式快速选择算法计算当前PU的能量和方向,根据能量和方向确定当前PU的预测模式,减少率失真代价的相关计算,达到降低编码复杂度和节省编码时间的目的。实验结果表明,在峰值信噪比(PSNR)损失仅为0.0390 dB的情况下,所提算法可以平均降低47.37%的编码时间。     

3D-HEVC合并模式快速判决方法研究

基于高效视频编码的3D视频编码（3D-HEVC）是目前正在研究的新一代3D视频编码标准。为降低3D-HEVC中模式选择的计算复杂度,根据非独立视点纹理图中合并模式采用率高的特点,该文提出了一种3D-HEVC合并模式快速判决方法。在B帧中,分析了当前编码单元（CU）与视点方向参考帧中参考块间编码模式的相关性;在P帧中,分析了位于相邻划分深度的CU间编码模式的相关性。根据分析的视点间和划分深度间的相关性设计快速判决条件,预判采用合并/合并-跳过模式编码的CU,判别出的CU在模式选择过程中只检查相关的候选预测模式,从而降低计算复杂度。实验结果表明,与3D-HEVC原始算法相比,该文算法能够在率失真性能损失很小的前提下,平均节省11.2%的总编码时间和25.4%的非独立视点纹理图的编码时间。      

A Fast Intra‐Prediction Method in HEVC Using Rate‐Distortion Estimation Based on Hadamard Transform

A fast intra‐prediction method is proposed for High Efficiency Video Coding (HEVC) using a fast intra‐mode decision and fast coding unit (CU) size decision. HEVC supports very sophisticated intra modes and a recursive quadtree‐based CU structure. To provide a high coding efficiency, the mode and CU size are selected in a rate‐distortion optimized manner. This causes a high computational complexity in the encoder, and, for practical applications, the complexity should be significantly reduced. In this paper, among the many predefined modes, the intra‐prediction mode is chosen without rate‐distortion optimization processes, instead using the difference between the minimum and second minimum of the rate‐distortion cost estimation based on the Hadamard transform. The experiment results show that the proposed method achieves a 49.04% reduction in the intra‐prediction time and a 32.74% reduction in the total encoding time with a nearly similar coding performance to that of HEVC test model 2.1.     

基于朴素贝叶斯分类的DVC-HEVC快速转码

分布式视频编码(DVC)与传统视频编码之间的转码为移动终端设备之间的低功耗视频通信提供了一种有效的实现思路。以DVC与HEVC转码为研究对象,利用DVC解码端信息,针对高效视频编码(HEVC)中复杂度极高的编码单元(CU)划分过程进行复杂度优化研究。在DVC解码端提取与CU划分相关的纹理复杂度、运动矢量及预测残差3种特征信息;在HEVC编码端基于朴素贝叶斯原理建立CU快速划分模型,模型生成后便可以通过输入特征信息对当前CU划分进行快速决策,避免大量率失真(RD)代价计算过程。实验结果表明,本方案在编码比特率略有上升的情况下大幅缩短了HEVC编码时间,平均下降幅度达到58.26%,且几乎不影响视频质量。     

基于灰度差值的HEVC快速帧间编码研究

针对新一代视频压缩编码标准HEVC计算复杂度较高的特点,利用视频序列间时域上的相关性,提出了一种基于灰度差值的编码单元快速划分策略.该策略根据当前编码块与参考块之间的灰度差值进行运动条件判决,在进行编码之前提前确定当前编码单元的编码深度信息,减少帧间预测编码的次数,从而有效地降低了编码端的计算复杂度.实验结果表明,该算法在编码效率和峰值信噪比(PSNR)损失都很小的情况下,和HM标准中的帧间预测算法相比,平均降低了50.18％的编码时间.     

CTU depth decision algorithms for HEVC: A survey

High Efficiency Video Coding (HEVC) surpasses its predecessors in encoding efficiency by introducing new coding tools at the cost of an increased encoding time-complexity. The Coding Tree Unit (CTU) is the main building block used in HEVC. In the HEVC standard, frames are divided into CTUs with the predetermined size of up to 64 × 64 pixels. Each CTU is then divided recursively into a number of equally sized square areas, known as Coding Units (CUs). Although this diversity of frame partitioning increases encoding efficiency, it also causes an increase in the time complexity due to the increased number of ways to find the optimal partitioning. To address this complexity, numerous algorithms have been proposed to eliminate unnecessary searches during partitioning CTUs by exploiting the correlation in the video. In this paper, existing CTU depth decision algorithms for HEVC are surveyed. These algorithms are categorized into two groups, namely statistics and machine learning approaches. Statistics approaches are further subdivided into neighboring and inherent approaches. Neighboring approaches exploit the similarity between adjacent CTUs to limit the depth range of the current CTU, while inherent approaches use only the available information within the current CTU. Machine learning approaches try to extract and exploit similarities implicitly. Traditional methods like support vector machines or random forests use manually selected features, while recently proposed deep learning methods extract features during training. Finally, this paper discusses extending these methods to more recent video coding formats such as Versatile Video Coding (VVC) and AOMedia Video 1(AV1).     

基于深层特征学习的高效率视频编码中帧内快速预测算法

高效视频编码(HEVC)标准相对于H.264/AVC标准提升了压缩效率,但由于引入的编码单元四叉树划分结构也使得编码复杂度大幅度提升。对此,该文提出一种针对HEVC帧内编码模式下编码单元(CU)划分表征矢量预测的多层特征传递卷积神经网络(MLFT-CNN),大幅度降低了视频编码复杂度。首先,提出融合CU划分结构信息的降分辨率特征提取模块;其次,改进通道注意力机制以提升特征的纹理表达性能;再次,设计特征传递机制,用高深度编码单元划分特征指导低深度编码单元的划分;最后建立分段特征表示的目标损失函数,训练端到端的CU划分表征矢量预测网络。实验结果表明,在不影响视频编码质量的前提下,该文所提算法有效地降低了HEVC的编码复杂度,与标准方法相比,编码复杂度平均下降了70.96%。     

基于多分支网络的深度图帧内编码单元快速划分算法

3维高效视频编码(3D-HEVC)标准是最新的3维(3D)视频编码标准,但由于其引入深度图编码技术导致编码复杂度大幅增加。其中,深度图帧内编码单元(CU)的四叉树划分占3D-HEVC编码复杂度的90%以上。对此,在3D-HEVC深度图帧内编码模式下,针对CU四叉树划分复杂度高的问题,该文提出一种基于深度学习的CU划分结构快速预测方案。首先,构建学习深度图CU划分结构信息的数据集;其次,搭建预测CU划分结构的多分支卷积神经网络(MB-CNN)模型,并利用构建的数据集训练MB-CNN模型;最后,将MB-CNN模型嵌入3D-HEVC的测试平台,通过直接预测深度图帧内编码模式下CU的划分结构来降低CU划分复杂度。与标准算法相比,编码复杂度平均降低了37.4%。实验结果表明,在不影响合成视点质量的前提下,该文所提算法有效地降低了3D-HEVC的编码复杂度。     

利用纹理结构的HEVC快速帧内模式选择算法

高性能视频编码（HEVC）是刚确立的最新一代的视频编码标准。对于帧内编码,HEVC最大的特点是采用了从64×64至8×8的编码单元划分和35种帧内预测模式,HEVC通过遍历所有的分块和帧内预测模式,选取最优的预测方式,这种帧内预测方式在提高预测精度的同时也大大增加了编码的计算复杂度。为了降低帧内预测的计算复杂度,HM2.0版以上的HEVC测试软件采用一种基于两步预测的快速帧内模式选择算法。在此基础上本文首先提出了一种基于纹理方向的快速粗选方案,减少参与计算的粗选模式数,进而提出基于结构相关的决策方法,利用同质图像区域的纹理相似性,减少参与率失真代价函数计算的候选模式数量,进一步降低了帧内预测计算复杂度。实验结果表明,本文所提出的快速算法,在保持编码质量基本不变的条件下,可以使基于两步预测的快速帧内模式选择时间缩短25%。      

CU encoding depth prediction,early CU splitting termination and fast mode decision for fast HEVC intra-coding

High Efficiency Video Coding (HEVC) is a new video coding standard achieving about a 50% bit rate reduction compared to the popular H.264/AVC High Profile with the same subjective reproduced video quality. Better coding efficiency is attained, however, at the cost of significantly increased encoding complexity. Therefore, fast encoding algorithms with little loss in coding efficiency is necessary for HEVC to be successfully adopted for real applications. In this paper we propose a fast encoding technique applicable to HEVC all intra encoding. The proposed fast encoding technique consists of coding unit (CU) search depth prediction, early CU splitting termination, and fast mode decision. In CU search depth prediction, the depth of encoded CU in the current coding tree unit (CTU) is limited to predicted range, which is mostly narrower than the full depth range. Early CU splitting skips mode search of its sub-CUs when rate distortion (RD) cost of current CU is below the estimated RD cost at the current CU depth. RD cost and encoded CU depth distribution of the collocated CTU of the previous frame are utilized both to predict the encoding CU depth search range and to estimate the RD cost for CU splitting termination. Fast mode decision reduces the number of candidate modes for full rate distortion optimized quantization on the basis of the low complexity costs computed in the preceding rough mode decision step. When all these three methods are applied, proposed fast HEVC intra encoding technique reduces the encoding time of the reference encoder by 57% on the average, with only 0.6% of coding efficiency loss in terms of Bjontegaard delta (BD) rate increase under the HEVC common test conditions.     

采用联合特征的H.266/VVC屏幕内容快速模式选择算法

新一代视频编码标准H.266/VCC针对屏幕内容编码引入帧内块复制、调色板等预测模式,在提高编码效率的同时也带来了巨大的计算复杂度。本文采用角点、水平/垂直活动度和平均颜色亮度差联合特征对屏幕内容视频帧和当前编码单元（Coding Unit, CU）进行分类,将视频帧分为低对比度屏幕视频帧和高对比度屏幕视频帧,将CU细分为自然内容背景CU、屏幕内容背景CU和屏幕内容前景CU,进而对CU模式进行选择性跳过,减少模式选择代价。实验结果显示,在全帧内的条件下,该算法与开启IBC和PLT的H.266/VVC VTM-7.3标准相比,在码率仅增加1.21%的情况下节省了26.41%的编码时间,降低了编码复杂度。      

Self-learning residual model for fast intra CU size decision in 3D-HEVC

As an extension of the High Efficiency Video Coding (HEVC) standard, 3D-HEVC requires to encode multiple texture views and depth maps, which inherits the same quad-tree coding structure as HEVC. Due to the distinct properties of texture views and depth maps, existing fast intra prediction approaches were presented for the coding of texture views and depth maps, respectively. To further reduce the coding complexity of 3D-HEVC, a self-learning residual model-based fast coding unit (CU) size decision approach is proposed for the intra coding of both texture views and depth maps. Residual signal, which is defined as the difference between the original luminance pixel and the optimal prediction luminance pixel, is firstly extracted from each CU. Since residue signal is strongly correlated with the optimal CU partition, it is used as the feature of each CU. Then, a self-learning residual model is established by intra feature learning, which iteratively learns the features of the previously encoded coding tree unit (CTU) generated by itself. Finally, a binary classifier is developed with the self-learning residual model to early terminate CU size decision of both texture views and depth maps. Experimental results show the proposed fast intra CU size decision approach achieves 33.3% and 49.3% encoding time reduction on average for texture views and depth maps with negligible loss of overall video quality, respectively.     

Fast intra-encoding algorithm for High Efficiency Video Coding

The emerging High Efficiency Video Coding (HEVC) standard provides equivalent subjective quality with about 50% bit rate reduction compared to the H.264/AVC High profile. However, the improvement of coding efficiency is obtained at the expense of increased computational complexity. This paper presents a fast intra-encoding algorithm for HEVC, which is composed of the following four techniques. Firstly, an early termination technique for coding unit (CU) depth decision is proposed based on the depth of neighboring CUs and the comparison results of rate distortion (RD) costs between the parent CU and part of its child CUs. Secondly, the correlation of intra-prediction modes between neighboring PUs is exploited to accelerate the intra-prediction mode decision for HEVC intra-coding and the impact of the number of mode candidates after the rough mode decision (RMD) process in HM is studied in our work. Thirdly, the TU depth range is restricted based on the probability of each TU depth and one redundant process is removed in the TU depth selection process based on the analysis of the HEVC reference software. Finally, the probability of each case for the intra-transform skip mode is studied to accelerate the intra-transform skip mode decision. Experimental results show that the proposed algorithm can provide about 50% time savings with only 0.5% BD-rate loss on average when compared to HM 11.0 for the Main profile all-intra-configuration. Parts of these techniques have been adopted into the HEVC reference software.     

Machine learning based fast H.264/AVC to HEVC transcoding exploiting block partition similarity

Video transcoding is to convert one compressed video stream to another. In this paper, a fast H.264/AVC to High Efficiency Video Coding (HEVC) transcoding method based on machine learning is proposed by considering the similarity between compressed streams, especially the block partition correlations, to reduce the computational complexity. This becomes possible by constructing three-level binary classifiers to predict quad-tree Coding Unit (CU) partition in HEVC. Then, we propose a feature selection algorithm to get representative features to improve predication accuracy of the classification. In addition, we propose an adaptive probability threshold determination scheme to achieve a good trade-off between low coding complexity and high compression efficiency during the CU depth prediction in HEVC. Extensive experimental results demonstrate the proposed transcoder achieves complexity reduction of 50.2% and 49.2% on average under lowdelay P main and random access configurations while the rate-distortion degradation is negligible. The proposed scheme is proved more effective as comparing with the state-of-the-art benchmarks.