3D-HEVC合并模式快速判决方法研究

基于高效视频编码的3D视频编码（3D-HEVC）是目前正在研究的新一代3D视频编码标准。为降低3D-HEVC中模式选择的计算复杂度,根据非独立视点纹理图中合并模式采用率高的特点,该文提出了一种3D-HEVC合并模式快速判决方法。在B帧中,分析了当前编码单元（CU）与视点方向参考帧中参考块间编码模式的相关性;在P帧中,分析了位于相邻划分深度的CU间编码模式的相关性。根据分析的视点间和划分深度间的相关性设计快速判决条件,预判采用合并/合并-跳过模式编码的CU,判别出的CU在模式选择过程中只检查相关的候选预测模式,从而降低计算复杂度。实验结果表明,与3D-HEVC原始算法相比,该文算法能够在率失真性能损失很小的前提下,平均节省11.2%的总编码时间和25.4%的非独立视点纹理图的编码时间。      

Early Merge mode decision algorithm for 3D-HEVC based on learning model

3D-high efficiency video coding (3D-HEVC) standard is an extension of HEVC.Though 3D-HEVC effectively improves the compression efficiency of 3D video,it also brings huge computational complexity.To greatly reduce the 3D-HEVC coding complexity,an early Merge mode decision approach was proposed.The residual signal that encoded by the Merge mode was firstly extracted as feature information.A learning model was established in terms of the residual signals of the coding unit (CU) in current frame that used early Merge mode as the optimal mode.Finally,the residual signal was extracted for the Merge mode of current CU,and the learning model was used to predict whether the Merge mode was the optimal mode or not.Experimental results show that the proposed early Merge mode decision approach reduces the coding times of 3D-HEVC texture views and depth maps about 41.9% and 24.3%,respectively,and the coding performance degradation is almost negligible.Compared with existing early Merge mode decision approaches,the proposed approach further reduces the coding time,and can be easily integrated into the 3D-HEVC test model due to its design simplicity.     

Fast encoder decision for texture coding in 3D-HEVC

As a 3D extension of the High Efficiency Video Coding (HEVC) standard, 3D-HEVC is developed to improve the coding efficiency of multi-view video. However, the improvement of the coding efficiency is obtained at the expense of a computational complexity increase. How to relieve the computational burden of the encoder is becoming a critical problem in applications. In this paper, a fast encoder decision algorithm to encode the dependent texture views is proposed, where two strategies to accelerate encoder decision by exploiting inter-view correlations are utilized. The first one is an early merge mode decision algorithm, and the second one is an early CU splitting termination algorithm. Experimental results show that the proposed algorithm can achieve 47.1% encoding time saving with overall 0.1% BD-rate reduction compared to HTM (3D-HEVC test model) version 7 under the common test condition (CTC). Both of the two strategies have been adopted into the 3D-HEVC reference software and enabled as a default encoding process under CTC.     

Fast depth map mode decision based on depth–texture correlation and edge classification for 3D-HEVC

The 3D extension of High Efficiency Video Coding (3D-HEVC) has been adopted as the emerging 3D video coding standard to support the multi-view video plus depth map (MVD) compression. In the joint model of 3D-HEVC design, the exhaustive mode decision is required to be checked all the possible prediction modes and coding levels to find the one with least rate distortion cost in depth map coding. Furthermore, new coding tools (such as depth-modeling mode (DMM) and segment-wise depth coding (SDC)) are exploited for the characteristics of depth map to improve the coding efficiency. These achieve the highest possible coding efficiency to code depth map, but also bring a significant computational complexity which limits 3D-HEVC from real-time applications. In this paper, we propose a fast depth map mode decision algorithm for 3D-HEVC by jointly using the correlation of depth map-texture video and the edge information of depth map. Since the depth map and texture video represent the same scene at the same time instant (they have the same motion characteristics), it is not efficient to use all the prediction modes and coding levels in depth map coding. Therefore, we can skip some specific prediction modes and depth coding levels rarely used in corresponding texture video. Meanwhile, the depth map is mainly characterized by sharp object edges and large areas of nearly constant regions. By fully exploiting these characteristics, we can skip some prediction modes which are rarely used in homogeneity regions based on the edge classification. Experimental results show that the proposed algorithm achieves considerable encoding time saving while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.     

Fast intra mode decision for depth coding in 3D-HEVC

The emergent 3D High Efficiency Video Coding (3D-HEVC) is an extension of the High Efficiency Video Coding (HEVC) standard for the compression of the multi-view texture videos plus depth maps format. Since depth maps have different statistical properties compared to texture video, various new intra tools have been added to 3D-HEVC depth coding. In current 3D-HEVC, new intra tools are utilized together with the conventional HEVC intra prediction modes for depth coding. This technique achieves the highest possible coding efficiency, but leads to an extremely high computational complexity which limits 3D-HEVC from practical applications. In this paper, we propose a fast intra mode decision algorithm for depth coding in 3D-HEVC. The basic idea of the proposed algorithm is to utilize the depth map characteristics to predict the current depth prediction mode and skip some specific depth intra modes rarely used in 3D-HEVC depth coding. Based on this analysis, two fast intra mode decision strategies are proposed including reduction of the number of conventional intra prediction modes, and simplification of depth modeling modes (DMMs). Experimental results demonstrate that the proposed algorithm can save 30 % coding runtime on average while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.     

Multi-class support vector machine-based fast algorithm for 3D-HEVC depth video intra coding

The recursive splitting process of largest coding unit (LCU) and the mode search process of coding unit imposed enormous computational complexity on encoder. A multi-class support vector machine-based (MSVM) fast coding unit (CU) size decision algorithm for 3D-HEVC depth video intra-coding was proposed. The algorithm included two steps: off-line training and fast CU size and mode decision. In the process of off-line training, a MSVM model was constructed, where the texture complexity of current LCU, the optimal partition depth of its spatial neighboring LCU and co-located LCU in texture video were treated as feature vectors, and the optimal partition depth of LCU was utilized as corresponding class label. In the process of fast CU size and mode decision, features of LCU were extracted before cod-ing a LCU, then, a MSVM model was used to predict the class label. Finally, the class label that represents the largest parti-tion depth of the current LCU was employed to terminate the CU recursive splitting process and CU mode search process. Experimental results show that the proposed algorithm saves the encoding time of 3D-HEVC by 35.91% on average, and the encoding time of depth video by 40.04% on average, with negligible rendered virtual view image degradation.     

Self-learning residual model for fast intra CU size decision in 3D-HEVC

As an extension of the High Efficiency Video Coding (HEVC) standard, 3D-HEVC requires to encode multiple texture views and depth maps, which inherits the same quad-tree coding structure as HEVC. Due to the distinct properties of texture views and depth maps, existing fast intra prediction approaches were presented for the coding of texture views and depth maps, respectively. To further reduce the coding complexity of 3D-HEVC, a self-learning residual model-based fast coding unit (CU) size decision approach is proposed for the intra coding of both texture views and depth maps. Residual signal, which is defined as the difference between the original luminance pixel and the optimal prediction luminance pixel, is firstly extracted from each CU. Since residue signal is strongly correlated with the optimal CU partition, it is used as the feature of each CU. Then, a self-learning residual model is established by intra feature learning, which iteratively learns the features of the previously encoded coding tree unit (CTU) generated by itself. Finally, a binary classifier is developed with the self-learning residual model to early terminate CU size decision of both texture views and depth maps. Experimental results show the proposed fast intra CU size decision approach achieves 33.3% and 49.3% encoding time reduction on average for texture views and depth maps with negligible loss of overall video quality, respectively.     

Segment-based view synthesis optimization scheme in 3D-HEVC

The 3D extension of high efficiency video coding (3D-HEVC) adopts a view synthesis optimization (VSO) technique to improve the quality of synthesized views for depth map coding. The exact synthesized view distortion change (SVDC) is calculated in VSO which in turn brings huge coding complexity to the 3D-HEVC encoder due to the real view synthesis process. This work presents a scheme aimed at reducing coding complexity of the SVDC calculating process in the 3D-HEVC encoder. It skips line segments of pixels with variable lengths based on information from both of the textures and depth maps in the SVDC calculation. Experimental results demonstrate that the proposed scheme can reduce the coding complexity without any significant loss in rate distortion performance for the synthesized views.     

Fast motion and disparity estimation for HEVC based 3D video coding

The emerging international standard for high efficiency video coding (HEVC) based 3D video coding (3D-HEVC) is an extension of HEVC. In the test model of 3D-HEVC, variable size motion estimation (ME) and disparity estimation (DE) are both employed to select the best coding mode for each treeblock in the encoding process. This technique achieves the highest possible coding efficiency, but it brings extremely high computational complexity which limits 3D-HEVC from practical applications. In this paper, a fast ME/DE algorithm based on inter-view and spatial correlations is proposed to reduce 3D-HEVC computational complexity. Since the multi-view videos represent the same scene with similar characteristic, there is a high correlation among the coding information from inter-view prediction. Besides, the homogeneous regions in texture video have a strong spatial correlation, and thus spatially neighboring treeblocks have similar coding information. Therefore, we can determine ME search range and skip some specific ME and DE rarely used in the previously coded view frames and spatially neighboring coding unit. Experimental results demonstrate that the proposed algorithm can significantly reduce computational complexity of 3D-HEVC encoding while maintaining almost the same rate-distortion performance.     

一种基于时域相关性的高性能视频编码快速帧间预测单元模式判决算法

为了降低高性能视频编码(HEVC)的编码计算复杂度,根据视频时域上高度相关性的特点,该文提出一种快速高性能视频编码(HEVC)帧间预测单元(PU)模式判决算法。分析了时域上相邻帧两帧相同位置编码单元(CU)的PU模式之间的相关性;同时,针对视频中可能存在对象运动,还分析了前一帧对应位置CU的周边CU与当前帧中当前CU间PU模式的相关性。根据分析的时域相关性,跳过当前CU中冗余的PU模式,从而降低编码复杂度。实验结果表明,在编码效率和峰值信噪比(PSNR)损失很小的情况下,在目前已有的HEVC快速帧间预测算法的基础上,进一步降低了31.30%的编码时间。     

Unified depth intra coding for 3D video extension of HEVC

With the development of high-efficiency video coding (HEVC), the newest video coding standard, 3D video extension of HEVC (3D-HEVC) has been actively investigated. Since 3D-HEVC uses multi-view texture and depth data for input, various coding tools have been added to HEVC. In 3D-HEVC, on top of the existing 35 HEVC intra modes, eight additional modes exist, which are specifically for depth coding. In this paper, we propose a unified depth intra coding method that incorporates such depth intra modes into the regular intra mode set. In particular, the most rarely used HEVC intra modes are replaced by depth intra modes. As a result, binarization for depth intra modes is removed. Furthermore, the most probable mode selection procedure is modified to consider the elimination of several angular intra modes. The proposed method is implemented and tested on 3D video HEVC test model version 7.0. Simulation results report 2.2 % synthesis gain under all-intra configuration.     

Low-Complexity Intra-Coding Scheme for HEVC

Compared with H.264/AVC, the latest video standard High Efficiency Video Coding (HEVC) known as H.265 improves the coding efficiency by adopting the quadtree splitting structure which is flexible in representing various textural and structural information in images. However, the computational complexity is dramatically increased, especially in the intra-mode decision process owing to supporting more partitions and modes. In this paper, we propose a low-complexity algorithm for HEVC intra-coding, which consists of a fast coding unit (CU) size decision (FCUSD) method and a fast prediction unit (PU) mode decision (FPUMD) method. In FCUSD, unnecessary CU sizes are skipped early according to the depth level of neighboring CUs and the rate distortion (RD) cost threshold derived from the former coded frame. In FPUMD, the PU mode and RD cost correlations between different depth levels are utilized to terminate unnecessary candidate modes. Experimental results demonstrate that the proposed algorithm can achieve about 50.99 % computational complexity reduction on average with 1.18 % BD-rate increase and 0.08 dB BD-psnr loss.     

Fast algorithm based on the sole- and multi-depth texture measurements for HEVC intra coding

In High Efficiency Video Coding (HEVC), intra coding plays an important role, but also involves huge computational complexity due to a flexible coding unit (CU) structure and a large number of prediction modes. This paper presents a fast algorithm based on the sole- and multi-depth texture measurements to reduce the complexity from CU size and prediction mode decisions. For the CU size decision, evaluation results in the CU coding with one and multiple depths are utilized to classify CUs into heterogeneous, homogeneous, depth-prominent and other ones. Fast CU size decisions are made for different kinds of CUs. For the prediction mode decision, the tendencies for different CU sizes are detected based on multiple depths. The number of searching modes is decreased adaptively for the CU size with fewer tendencies. Experimental results show the proposed algorithm by off-line training reduces 53.32% computational complexity, with 1.47% bit-rate increasing.     

3D-HEVC深度图帧内编码快速算法

编码3D视频的3D-HEVC编码标准采用多视点加深度图的编码格式,新增的深度信息使编码复杂度剧增。本文针对编码块(Coding Unit,CU)的四叉树分割模型和帧内预测模式,提出了深度图帧内编码的快速算法。用Otsu’s算子计算当前CU的最大类间方差值,判断当前CU是否平坦,对平坦CU终止四叉树分割和减少帧内模式的遍历数目。根据子CU与上一层CU的相似性,利用已编码的上一层CU对提前终止CU分割算法做优化。本算法与原始3D-HEVC算法相比减少40.1%的编码时间,而合成视点的质量几乎无变化。      

A novel fast intra mode decision for versatile video coding

The latest video coding standard Versatile Video Coding (VVC) obtains superior coding efficiency compared to the High Efficiency Video Coding (HEVC), which is achieved by incorporating more effective and complex new coding tools. In this paper, we propose a novel fast intra mode decision algorithm for VVC, including following two strategies: (1) the correlation between the optimal modes of the adjacent blocks and the modes selected in the rough modes decision (RMD) process is analyzed and applied to reduce the modes in the candidate list; (2) modes in the candidate list are sorted in ascending order according to the modes’ cost calculated in the RMD process. An early termination method is proposed for terminating the optimal prediction mode decision process based on this new order early. These two strategies are incorporated into intra coding to reduce the coding complexity. Since these two strategies do not add any additional computational complexity, the proposed fast algorithm can achieve more complexity reduction. The experimental results show that the complexity reduction of the proposed algorithm is up to 44.74% compared to VVC reference software VTM2.0, and averagely 30.59% encoding time saving with 0.86% BDBR increase.     

纹理-深度图共同优化的3D-HEVC帧内快速算法

随着立体及3D视频需求的日益增多,针对3D视频编码方法的研究受到越来越多的关注。3D-HEVC编码标准对采用纹理和深度图格式融合的3D视频进行编码,由于加入了深度图编码,因此新增了深度图编码模式、组件间预测和分段直流编码等技术,使其编码复杂度急剧升高。为了减少3D-HEVC的编码时间,本文提出了针对纹理图和深度图的编码单元（Coding Unit,CU）尺寸提前决策快速算法。利用梯度矩阵和作为当前CU和子CU复杂度的判断依据,将CU分为三类：不划分CU(Non-Split Coding Unit,NSCU)、直接划分CU(Split Coding Unit,SCU)以及普通CU。对NSCU,跳过小尺寸的帧内预测过程;对SCU,直接跳过当前CU的帧内预测过程;对普通CU,执行原平台操作。实验结果表明,与原始平台相比,本文算法在合成视点质量基本不变的情况下,平均减少40.92%的编码时间;与最新的联合纹理-深度图优化的3D-HEVC快速算法相比,可以在质量相当的情况下减少更多的编码时间。     

Fast intra-encoding algorithm for High Efficiency Video Coding

The emerging High Efficiency Video Coding (HEVC) standard provides equivalent subjective quality with about 50% bit rate reduction compared to the H.264/AVC High profile. However, the improvement of coding efficiency is obtained at the expense of increased computational complexity. This paper presents a fast intra-encoding algorithm for HEVC, which is composed of the following four techniques. Firstly, an early termination technique for coding unit (CU) depth decision is proposed based on the depth of neighboring CUs and the comparison results of rate distortion (RD) costs between the parent CU and part of its child CUs. Secondly, the correlation of intra-prediction modes between neighboring PUs is exploited to accelerate the intra-prediction mode decision for HEVC intra-coding and the impact of the number of mode candidates after the rough mode decision (RMD) process in HM is studied in our work. Thirdly, the TU depth range is restricted based on the probability of each TU depth and one redundant process is removed in the TU depth selection process based on the analysis of the HEVC reference software. Finally, the probability of each case for the intra-transform skip mode is studied to accelerate the intra-transform skip mode decision. Experimental results show that the proposed algorithm can provide about 50% time savings with only 0.5% BD-rate loss on average when compared to HM 11.0 for the Main profile all-intra-configuration. Parts of these techniques have been adopted into the HEVC reference software.     

Sum-of-gradient based fast intra coding in 3D-HEVC for depth map sequence (SOG-FDIC)

As the latest video coding standard for multi-view plus depth video, 3D-HEVC yields high coding efficiency but at the cost of heavy computational complexity. To reduce the computational complexity, a fast intra coding algorithm based on sum-of-gradient criterion for depth map coding in 3D-HEVC, named SOG-FDIC, is proposed in this paper. Based on the observation that DMM modes and smaller partitioning sizes are rarely used in flat region, sum of gradient is presented to determine whether the current block belongs to the flat region so as to skip unnecessary checking of DMMs and smaller partitioning sizes. Experimental results show that the proposed algorithm can save about 21.8% coding time while keeping almost the same coding efficiency and the reconstructed video quality of depth maps and synthesized views, compared with the original 3D-HEVC. Moreover, it has been verified that the proposed method outperforms the state-of-the-art methods.     

3D-HEVC深度图帧内预测快速编码算法

三维高效视频编码在产生了高效的编码效率的同时也是以大量的计算复杂性作为代价的。因此为了降低计算的复杂度,本文提出了一种基于深度学习网络的边缘检测的3D-HEVC深度图帧内预测快速算法。算法中首先使用整体嵌套边缘检测网络对深度图进行边缘检测,而后使用最大类间方差法将得到的概率边缘图进行二值化处理,得到显著性的边缘区域。最后针对处于不同区域的不同尺寸的预测单元,设计了不同的优化方法,通过跳过深度建模模式和其他某些不必要的模式来降低深度图帧内预测的模式选择的复杂度,最终达到减少深度图的编码复杂度的目的。经过实验仿真的验证,本文提出的算法与原始的编码器算法相比,平均总编码时间可减少35%左右,且深度图编码时间平均大约可减少42%,而合成视点的平均比特率仅增加了0.11%。即本文算法在可忽略的质量损失下,达到降低编码时间的目的。     

一种快速的SVC多模式决策方法

H.264/AVC SVC为了提供一种可伸缩的视频编码解决方案,增加了许多宏块模式的组合,从而导致模式决策的计算量大幅度增加。本文提出了一种面向SVC的快速多模式决策算法,该方法首先利用模式在层间、帧间和空间的相关性,决定当前宏块的起始搜索模式,然后利用相邻运动向量的差异性决定是否合并或拆分当前分块进行细化搜索。从测试结果来看,本算法在率失真性能不降低的情况下,令模式决策的运算速度提高了数倍。