Early merge mode decision for texture coding in 3D-HEVC

As the upcoming 3D video coding standard, high efficiency video coding (HEVC) based 3D video coding (3D-HEVC) has been drafted. In 3D-HEVC, the computational complexity of mode decision process is significantly high due to exhaustive modes’ checks for coding units (CU) derived from recursive quad-tree partitioning. In this paper, we propose an early merge mode decision method for complexity reduction of dependent texture views. First, inter-view correlation and hierarchical depth correlation of coding modes are separately analyzed for B frame and P frame. Then, conditions to early determine merge mode coded CUs are derived based on the correlations. All of the early determined CUs only check merge modes in the mode decision process, which brings considerable complexity reduction. Experimental results demonstrate that the proposed method can achieve average 20.4% of encoding time saving for dependent texture views with negligible rate distortion performance loss.     

Fast encoder decision for texture coding in 3D-HEVC

As a 3D extension of the High Efficiency Video Coding (HEVC) standard, 3D-HEVC is developed to improve the coding efficiency of multi-view video. However, the improvement of the coding efficiency is obtained at the expense of a computational complexity increase. How to relieve the computational burden of the encoder is becoming a critical problem in applications. In this paper, a fast encoder decision algorithm to encode the dependent texture views is proposed, where two strategies to accelerate encoder decision by exploiting inter-view correlations are utilized. The first one is an early merge mode decision algorithm, and the second one is an early CU splitting termination algorithm. Experimental results show that the proposed algorithm can achieve 47.1% encoding time saving with overall 0.1% BD-rate reduction compared to HTM (3D-HEVC test model) version 7 under the common test condition (CTC). Both of the two strategies have been adopted into the 3D-HEVC reference software and enabled as a default encoding process under CTC.     

Selective Disparity Estimation and Variable Size Motion Estimation Based on Motion Homogeneity for Multi-View Coding

Multi-view video coding (MVC) is an ongoing standard in which variable size disparity estimation (DE) and motion estimation (ME) are both employed to select the best coding mode for each macroblock (MB). This technique achieves the highest possible coding efficiency, but it results in extremely large encoding time which obstructs it from practical use. In this paper, a fast DE and ME algorithm based on motion homogeneity is proposed to reduce MVC computational complexity. The basic idea of the method is to utilize the spatial property of motion field in prediction where DE and variable size ME are needed, and only in these regions DE and variable size ME are enabled. The motion field is generated by the corresponding motion vectors (MVs) in spatial window. Simulation results show that the proposed algorithm can save 63% average computational complexity, with negligible loss of coding efficiency.      

Early SKIP mode decision for MVC using inter-view correlation

In the joint multiview video model (JMVM) proposed by JVT, the variable block-size motion estimation (ME) and disparity estimation (DE) have been employed to determine the best coding mode for each macroblock (MB). These give a high coding efficiency for multiview video coding (MVC), however, they cause a very high computational complexity in encoding system. This paper proposes to reduce the complexities of the ME and DE processes with an early SKIP mode decision algorithm based on the analysis of prediction mode distribution regarding the corresponding MBs in the neighbor view. In this method, the mode decision procedures of most of MBs can be early terminated, and thus much of computation for ME and DE can be greatly reduced. Simulation results demonstrate that our algorithm can achieve computational saving of 46–57% (depending on the tested sequences) with no significant loss of rate-distortion performance.     

3D-HEVC合并模式快速判决方法研究

基于高效视频编码的3D视频编码（3D-HEVC）是目前正在研究的新一代3D视频编码标准。为降低3D-HEVC中模式选择的计算复杂度,根据非独立视点纹理图中合并模式采用率高的特点,该文提出了一种3D-HEVC合并模式快速判决方法。在B帧中,分析了当前编码单元（CU）与视点方向参考帧中参考块间编码模式的相关性;在P帧中,分析了位于相邻划分深度的CU间编码模式的相关性。根据分析的视点间和划分深度间的相关性设计快速判决条件,预判采用合并/合并-跳过模式编码的CU,判别出的CU在模式选择过程中只检查相关的候选预测模式,从而降低计算复杂度。实验结果表明,与3D-HEVC原始算法相比,该文算法能够在率失真性能损失很小的前提下,平均节省11.2%的总编码时间和25.4%的非独立视点纹理图的编码时间。      

Adaptive fractional motion and disparity estimation skipping in MV-HEVC

MV-HEVC can efficiently compress multiview video data captured from different viewpoints. To achieve high coding efficiency, it consists of not only inter coding but also interview coding. The inter coding includes a motion estimation (ME) process that reduces temporal redundancies between consecutive frames, and the interview coding performs a disparity estimation (DE) that reduces interview redundancies between neighboring views. As a result, MV-HEVC needs high encoding complexity to perform both ME and DE. In order to reduce the complexity, this paper proposes an adaptive fractional ME and DE skipping method in a partitioned inter prediction unit (PU) mode, based on a result of a 2 N × 2 N inter PU coding. Experimental results show that the proposed method efficiently reduces the encoding complexity with negligible coding loss, compared to conventional methods.     

Fast intra mode decision for depth coding in 3D-HEVC

The emergent 3D High Efficiency Video Coding (3D-HEVC) is an extension of the High Efficiency Video Coding (HEVC) standard for the compression of the multi-view texture videos plus depth maps format. Since depth maps have different statistical properties compared to texture video, various new intra tools have been added to 3D-HEVC depth coding. In current 3D-HEVC, new intra tools are utilized together with the conventional HEVC intra prediction modes for depth coding. This technique achieves the highest possible coding efficiency, but leads to an extremely high computational complexity which limits 3D-HEVC from practical applications. In this paper, we propose a fast intra mode decision algorithm for depth coding in 3D-HEVC. The basic idea of the proposed algorithm is to utilize the depth map characteristics to predict the current depth prediction mode and skip some specific depth intra modes rarely used in 3D-HEVC depth coding. Based on this analysis, two fast intra mode decision strategies are proposed including reduction of the number of conventional intra prediction modes, and simplification of depth modeling modes (DMMs). Experimental results demonstrate that the proposed algorithm can save 30 % coding runtime on average while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.     

Efficient Inter Prediction Mode Decision Method for Fast Motion Estimation in High Efficiency Video Coding

High Efficiency Video Coding (HEVC) is the most recent video coding standard to achieve a higher coding performance than the previous H.264/AVC. In order to accomplish this improved coding performance, HEVC adopted several advanced coding tools; however, these cause heavy computational complexity. Similar to previous video coding standards, motion estimation (ME) of HEVC requires the most computational complexity; this is because ME is conducted for three inter prediction modes — namely, uniprediction in list 0, uniprediction in list 1, and biprediction. In this paper, we propose an efficient inter prediction mode (EIPM) decision method to reduce the complexity of ME. The proposed EIPM method computes the priority of all inter prediction modes and performs ME only on a selected inter prediction mode. Experimental results show that the proposed method reduces computational complexity arising from ME by up to 51.76% and achieves near similar coding performance compared to HEVC test model version 10.1.     

3D-HEVC深度图帧内预测快速编码算法

三维高效视频编码在产生了高效的编码效率的同时也是以大量的计算复杂性作为代价的。因此为了降低计算的复杂度,本文提出了一种基于深度学习网络的边缘检测的3D-HEVC深度图帧内预测快速算法。算法中首先使用整体嵌套边缘检测网络对深度图进行边缘检测,而后使用最大类间方差法将得到的概率边缘图进行二值化处理,得到显著性的边缘区域。最后针对处于不同区域的不同尺寸的预测单元,设计了不同的优化方法,通过跳过深度建模模式和其他某些不必要的模式来降低深度图帧内预测的模式选择的复杂度,最终达到减少深度图的编码复杂度的目的。经过实验仿真的验证,本文提出的算法与原始的编码器算法相比,平均总编码时间可减少35%左右,且深度图编码时间平均大约可减少42%,而合成视点的平均比特率仅增加了0.11%。即本文算法在可忽略的质量损失下,达到降低编码时间的目的。     

Early Merge mode decision algorithm for 3D-HEVC based on learning model

3D-high efficiency video coding (3D-HEVC) standard is an extension of HEVC.Though 3D-HEVC effectively improves the compression efficiency of 3D video,it also brings huge computational complexity.To greatly reduce the 3D-HEVC coding complexity,an early Merge mode decision approach was proposed.The residual signal that encoded by the Merge mode was firstly extracted as feature information.A learning model was established in terms of the residual signals of the coding unit (CU) in current frame that used early Merge mode as the optimal mode.Finally,the residual signal was extracted for the Merge mode of current CU,and the learning model was used to predict whether the Merge mode was the optimal mode or not.Experimental results show that the proposed early Merge mode decision approach reduces the coding times of 3D-HEVC texture views and depth maps about 41.9% and 24.3%,respectively,and the coding performance degradation is almost negligible.Compared with existing early Merge mode decision approaches,the proposed approach further reduces the coding time,and can be easily integrated into the 3D-HEVC test model due to its design simplicity.     

基于时空相关性的HEVC帧间模式决策快速算法

新一代的高效率视频编码标准HEVC采用编码树单元（CTU）四叉树划分技术和多达10种的帧间预测单元（PU）模式,有效地提高了编码压缩效率,但也极大地增加了编码计算复杂度。为了减少编码单元（CU）的划分次数和候选帧间PU模式个数,提出了一种基于时空相关性的帧间模式决策快速算法。首先,利用当前CTU与参考帧中相同位置CTU、当前帧中相邻CTU的深度信息时空相关性,有效预测当前CTU的深度范围。然后,通过分析当前CU与其父CU之间的最佳PU模式空间相关性,以及利用当前CU已估计PU模式的率失真代价,跳过当前CU的冗余帧间PU模式。实验结果表明,提出的算法与HEVC测试模型（HM）相比,在不同编码配置下降低了52%左右的编码时间,同时保持了良好的编码率失真性能;与打开快速算法选项的HM相比,所提算法进一步降低了30%左右的编码时间。     

Sum-of-gradient based fast intra coding in 3D-HEVC for depth map sequence (SOG-FDIC)

As the latest video coding standard for multi-view plus depth video, 3D-HEVC yields high coding efficiency but at the cost of heavy computational complexity. To reduce the computational complexity, a fast intra coding algorithm based on sum-of-gradient criterion for depth map coding in 3D-HEVC, named SOG-FDIC, is proposed in this paper. Based on the observation that DMM modes and smaller partitioning sizes are rarely used in flat region, sum of gradient is presented to determine whether the current block belongs to the flat region so as to skip unnecessary checking of DMMs and smaller partitioning sizes. Experimental results show that the proposed algorithm can save about 21.8% coding time while keeping almost the same coding efficiency and the reconstructed video quality of depth maps and synthesized views, compared with the original 3D-HEVC. Moreover, it has been verified that the proposed method outperforms the state-of-the-art methods.     

Multi-class support vector machine-based fast algorithm for 3D-HEVC depth video intra coding

The recursive splitting process of largest coding unit (LCU) and the mode search process of coding unit imposed enormous computational complexity on encoder. A multi-class support vector machine-based (MSVM) fast coding unit (CU) size decision algorithm for 3D-HEVC depth video intra-coding was proposed. The algorithm included two steps: off-line training and fast CU size and mode decision. In the process of off-line training, a MSVM model was constructed, where the texture complexity of current LCU, the optimal partition depth of its spatial neighboring LCU and co-located LCU in texture video were treated as feature vectors, and the optimal partition depth of LCU was utilized as corresponding class label. In the process of fast CU size and mode decision, features of LCU were extracted before cod-ing a LCU, then, a MSVM model was used to predict the class label. Finally, the class label that represents the largest parti-tion depth of the current LCU was employed to terminate the CU recursive splitting process and CU mode search process. Experimental results show that the proposed algorithm saves the encoding time of 3D-HEVC by 35.91% on average, and the encoding time of depth video by 40.04% on average, with negligible rendered virtual view image degradation.     

Fast prediction unit selection method for HEVC intra prediction based on salient regions

In order to reduce the computational complexity of the high efficiency video coding(HEVC) standard, a new algorithm for HEVC intra prediction, namely, fast prediction unit(PU) size selection method for HEVC based on salient regions is proposed in this paper. We first build a saliency map for each largest coding unit(LCU) to reduce its texture complexity. Secondly, the optimal PU size is determined via a scheme that implements an information entropy comparison among sub-blocks of saliency maps. Finally, we apply the partitioning result of saliency map on the original LCUs, obtaining the optimal partitioning result. Our algorithm can determine the PU size in advance to the angular prediction in intra coding, reducing computational complexity of HEVC. The experimental results show that our algorithm achieves a 37.9% reduction in encoding time, while producing a negligible loss in Bjontegaard delta bit rate(BDBR) of 0.62%.     

基于纹理相似性的高效视频编码帧间预测优化算法

为了提高高效率视频编码(HEVC)的编码效率,该文对帧间预测算法进行优化,提出一种基于纹理相似性的快速深度判决算法。随着视频分辨率的提高,视频序列中的空域冗余也随之增加,HEVC通过增加编码块尺寸来提高预测效率,代价就是编码复杂度的急剧增加。通过对视频序列分析可知,图像中的平滑区域与细节区域在相邻帧中存在很大的相关性。该文利用相邻已编码帧的相邻编码块的深度信息,来快速判决当前待编码块的深度信息。对于平滑区域,快速判决待编码最大编码单元的最大深度,以减少小块编码单元的模式判决;对于细节区域,快速判决待编码最大编码单元的最小深度,以减少大块编码单元的模式判决。实验结果表明,与原始的HEVC算法相比,该文所提算法编码比特率平均增加约0.13%以内,PSNR的平均降幅为0.09 dB,算法运行时间平均减少了约50%。     

基于纹理方向和空域相关的HEVC帧内快速编码算法

为了降低高效视频编码(HEVC)的复杂度,提出了一种基于纹理方向和空域相关性的帧内快速编码算法。一方面,利用相邻编码单元(CU,coding unit)的最佳分割尺寸确定相关性的权重,并根据已编码CU的率失真代价值定义了CU分割尺寸的相关性因子,通过比较该因子提前终止CU分割;另一方面,利用Sobel算子求取预测单元(PU,prediction unit)子块的纹理方向,通过判定其纹理方向显著性确定相应的候选模式集,然后根据PU大小对所得的预测模式修正处理,最后遍历这些候选模式选取最优模式。实验结果表明:本文算法相对于原始HEVC编码方法,在全I帧模式下编码时间平均减少36.84%,BDBR(Bjntegaard delta bit rate)上升约0.81%,BDPSNR(Bjntegaard delta peak signal-to-noise rate)降低了0.047dB。     

Fast depth map mode decision based on depth–texture correlation and edge classification for 3D-HEVC

The 3D extension of High Efficiency Video Coding (3D-HEVC) has been adopted as the emerging 3D video coding standard to support the multi-view video plus depth map (MVD) compression. In the joint model of 3D-HEVC design, the exhaustive mode decision is required to be checked all the possible prediction modes and coding levels to find the one with least rate distortion cost in depth map coding. Furthermore, new coding tools (such as depth-modeling mode (DMM) and segment-wise depth coding (SDC)) are exploited for the characteristics of depth map to improve the coding efficiency. These achieve the highest possible coding efficiency to code depth map, but also bring a significant computational complexity which limits 3D-HEVC from real-time applications. In this paper, we propose a fast depth map mode decision algorithm for 3D-HEVC by jointly using the correlation of depth map-texture video and the edge information of depth map. Since the depth map and texture video represent the same scene at the same time instant (they have the same motion characteristics), it is not efficient to use all the prediction modes and coding levels in depth map coding. Therefore, we can skip some specific prediction modes and depth coding levels rarely used in corresponding texture video. Meanwhile, the depth map is mainly characterized by sharp object edges and large areas of nearly constant regions. By fully exploiting these characteristics, we can skip some prediction modes which are rarely used in homogeneity regions based on the edge classification. Experimental results show that the proposed algorithm achieves considerable encoding time saving while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.     

Fast sample adaptive offset algorithm for 360-degree video coding

360-degree video is becoming popular with the development of the virtual reality(VR) technology in recent years. For this kind of video, pictures are projected to a two-dimensional plane. Standard video encoders are used for compression. Due to the projection mapping of the spherical video to the two-dimensional plane video, many conventional coding tools can be adjusted properly for better performance. As the 360-degree videos are high resolution videos, usually 8K resolution videos, the high complexity for encoding 360-degree videos is a problem to be solved urgently, especially for the latest video standard H.265/HEVC. Sample adaptive offset (SAO) is a new tool adopted in H.265/HEVC. It can reduce ringing artifacts and achieve higher coding efficiency. But it also introduces high computational complexity, especially on high resolution videos. In this paper, a fuzzy control based fast SAO method is proposed to reduce the high computational complexity in SAO encoding process according to the characteristics of the ERP format video. Compared with the original algorithm in the reference software HM 16.14, experimental results show that the proposed method can reduce about 73% SAO encoding time on average under the common test conditions with negligible loss.     

Content-adaptive motion estimation algorithm for coarse-grain SVC

A joint model of scalable video coding (SVC) uses exhaustive mode and motion searches to select the best prediction mode and motion vector for each macroblock (MB) with high coding efficiency at the cost of computational complexity. If major characteristics of a coding MB such as the complexity of the prediction mode and the motion property can be identified and used in adjusting motion estimation (ME), one can design an algorithm that can adapt coding parameters to the video content. This way, unnecessary mode and motion searches can be avoided. In this paper, we propose a content-adaptive ME for SVC, including analyses of mode complexity and motion property to assist mode and motion searches. An experimental analysis is performed to study interlayer and spatial correlations in the coding information. Based on the correlations, the motion and mode characteristics of the current MB are identified and utilized to adjust each step of ME at the enhancement layer including mode decision, search-range selection, and prediction direction selection. Experimental results show that the proposed algorithm can significantly reduce the computational complexity of SVC while maintaining nearly the same rate distortion performance as the original encoder.     

基于邻近值的HEVC帧内预测优化

高性能视频编码(High Efficiency Video Coding,HEVC)标准中的帧内编码模式利用当前帧中像素点之间的空间相关性作出有效预测。为了解决待编码像素远离参考像素时预测不准确的问题,提出了一种基于邻近值的HEVC帧内预测优化算法。该算法的主要思想是,对于当前像素,先根据传统HEVC帧内编码方法得到其预测值,再使用该像素点左边、左上、上边位置的修正值以及该像素本身的预测值对该预测值进行修正。因为将当前像素与周围像素的相关性进行了有效的数学建模,所以HEVC帧内编码性能得到了提升。实验结果显示,所提算法与HEVC标准相比,最高节省了2.7%的码率,平均节省的码率为1.3%。