基于多分支网络的深度图帧内编码单元快速划分算法

3维高效视频编码(3D-HEVC)标准是最新的3维(3D)视频编码标准,但由于其引入深度图编码技术导致编码复杂度大幅增加。其中,深度图帧内编码单元(CU)的四叉树划分占3D-HEVC编码复杂度的90%以上。对此,在3D-HEVC深度图帧内编码模式下,针对CU四叉树划分复杂度高的问题,该文提出一种基于深度学习的CU划分结构快速预测方案。首先,构建学习深度图CU划分结构信息的数据集;其次,搭建预测CU划分结构的多分支卷积神经网络(MB-CNN)模型,并利用构建的数据集训练MB-CNN模型;最后,将MB-CNN模型嵌入3D-HEVC的测试平台,通过直接预测深度图帧内编码模式下CU的划分结构来降低CU划分复杂度。与标准算法相比,编码复杂度平均降低了37.4%。实验结果表明,在不影响合成视点质量的前提下,该文所提算法有效地降低了3D-HEVC的编码复杂度。     

3D-HEVC深度视频编码MERGE模式快速选择

针对3D-HEVC中多视点深度视频编码进行研究,提出了一种对深度视频参考视点的MERGE模式快速选择算法。考虑到多视点深度视频相邻视点间相关性和MVD格式中深度视频与彩色视频间相关性,在两个方向上的参考CU采用相同编码深度时,借鉴深度视频主视点PU模式信息以加快当前CU帧间模式选择速度。在深度主视点采用MERGE模式时,当前视点跳过其他的预测模式而直接采用MERGE模式。实验结果表明,在码率和图像质量基本不变的情况下,提出的算法可以有效地提高帧间模式选择速度。     

结合预处理的深度视频帧内快速编码算法

多视点彩色加深度(MVD)视频是三维(3D)视频的 主流格式。在3D高效视频编码中,深度视频帧内编码 具有较高的编码复杂度;深度估计软件获取的深度视频由于不够准确会使深度图平坦 区域纹理增加, 从而进一步增加帧内编码复杂度。针对以上问题,本文提出了一种联合深度处理的深度视频 帧内低复杂度 编码算法。首先,在编码前对深度视频进行预处理,减少由于深度图不准确而出现的纹理信 息;其次,运 用反向传播神经网络(BPNN,backpropagation neural network)预测最大编码单元 (LCU,la rgest coding unit)的最大划分深度;最后联合深度视频的边缘信 息及对应的彩色LCU最大划分深度进行CU提前终止划分和快速模式选取。实验结果表明, 本文算法在保证 虚拟视点质量的前提下,BDBR下降0.33% ,深度视频编码时间平均节省50.63%。     

3D-HEVC深度图像帧内编码单元划分快速算法

针对3维高性能视频编码(3D-HEVC)中深度图像帧内编码单元(Coding Unit, CU)划分复杂度高的问题,该文提出一种基于角点和彩色图像的自适应快速CU划分算法。首先利用角点算子,并根据量化参数选取一定数目的角点,以此进行CU的预划分;然后联合彩色图像的CU划分对预划分的CU深度级进行调整;最后依据调整后的CU深度级,缩小当前CU的深度级范围。实验结果表明,与原始3D-HEVC的算法相比,该文所提算法平均减少了约63%的编码时间;与只基于彩色图像的算法相比,该文的算法减少了约13%的编码时间,同时降低了约3%的平均比特率,有效地提高了编码效率。     

纹理-深度图共同优化的3D-HEVC帧内快速算法

随着立体及3D视频需求的日益增多,针对3D视频编码方法的研究受到越来越多的关注。3D-HEVC编码标准对采用纹理和深度图格式融合的3D视频进行编码,由于加入了深度图编码,因此新增了深度图编码模式、组件间预测和分段直流编码等技术,使其编码复杂度急剧升高。为了减少3D-HEVC的编码时间,本文提出了针对纹理图和深度图的编码单元（Coding Unit,CU）尺寸提前决策快速算法。利用梯度矩阵和作为当前CU和子CU复杂度的判断依据,将CU分为三类：不划分CU(Non-Split Coding Unit,NSCU)、直接划分CU(Split Coding Unit,SCU)以及普通CU。对NSCU,跳过小尺寸的帧内预测过程;对SCU,直接跳过当前CU的帧内预测过程;对普通CU,执行原平台操作。实验结果表明,与原始平台相比,本文算法在合成视点质量基本不变的情况下,平均减少40.92%的编码时间;与最新的联合纹理-深度图优化的3D-HEVC快速算法相比,可以在质量相当的情况下减少更多的编码时间。     

采用灰度共生矩阵进行深度预判的3D-HEVC深度图帧内快速编码算法

针对3D视频的3D-HEVC编码标准以多视点纹理视频和深度视频格式进行编码,其深度图编码仍延续纹理视频编码的模式和编码尺寸遍历选择,使得3D-HEVC的编码复杂度居高不下。本文针对深度图帧内预测编码,采用灰度共生矩阵对深度图中的CTU进行计算,统计并分析其矩阵中非零值个数与CTU分割深度的关系,根据非零值个数分布规律,设定阈值,使得帧内编码时可以预判编码模块的分割深度,从而选择性跳过部分不同深度CU的帧内预测过程。经过HTM16.0测试平台的检验,本算法在全帧内编码模式下,测试序列合成视点比特率仅增加0.08%的同时,平均节省了16.8%的编码时间,与其他同类较新算法在HTM16.0平台上的性能比较也有一定的优势。      

3D-HEVC深度图帧内编码快速算法

编码3D视频的3D-HEVC编码标准采用多视点加深度图的编码格式,新增的深度信息使编码复杂度剧增。本文针对编码块(Coding Unit,CU)的四叉树分割模型和帧内预测模式,提出了深度图帧内编码的快速算法。用Otsu’s算子计算当前CU的最大类间方差值,判断当前CU是否平坦,对平坦CU终止四叉树分割和减少帧内模式的遍历数目。根据子CU与上一层CU的相似性,利用已编码的上一层CU对提前终止CU分割算法做优化。本算法与原始3D-HEVC算法相比减少40.1%的编码时间,而合成视点的质量几乎无变化。      

一种基于时域相关性的高性能视频编码快速帧间预测单元模式判决算法

为了降低高性能视频编码(HEVC)的编码计算复杂度,根据视频时域上高度相关性的特点,该文提出一种快速高性能视频编码(HEVC)帧间预测单元(PU)模式判决算法。分析了时域上相邻帧两帧相同位置编码单元(CU)的PU模式之间的相关性;同时,针对视频中可能存在对象运动,还分析了前一帧对应位置CU的周边CU与当前帧中当前CU间PU模式的相关性。根据分析的时域相关性,跳过当前CU中冗余的PU模式,从而降低编码复杂度。实验结果表明,在编码效率和峰值信噪比(PSNR)损失很小的情况下,在目前已有的HEVC快速帧间预测算法的基础上,进一步降低了31.30%的编码时间。     

面向三维高效视频编码的深度图错误隐藏

基于多视点视频序列视点内、视点间存在的相关性,并结合视点间运动矢量共享技术,该文提出一种面向3维高效视频编码中深度序列传输丢包的错误隐藏算法。首先,根据3D高效视频编码(3D-HEVC)的分层B帧预测(HBP)结构和深度图纹理特征,将深度图丢失块分成运动块和静止块;然后,对于受损运动块,使用结合纹理结构的外边界匹配准则来选择相对最优的运动/视差矢量进行基于位移矢量补偿的错误掩盖,而对受损静止块采用参考帧直接拷贝进行快速错误隐藏;最后,使用参考帧拆分重组来获取新的运动/视差补偿块对修复质量较差的重建块进行质量提升。实验结果表明:相较于近年提出的对比算法,该文算法隐藏后的深度帧平均峰值信噪比(PSNR)能提升0.25～2.03 dB,结构相似度测量值(SSIM)能提升0.001～0.006,且修复区域的主观视觉质量与原始深度图更接近。     

基于单深度帧内模式的3D-HEVC深度视频信息隐藏算法

针对三维(3D)视频系统中深度视频的安全问题,利 用一种全新的信息嵌入载体,提出 一种基于单深度帧内模式(single depth intra mode)的3D高效视频编码(3D-HEVC)深度视 频信息隐藏算法。首先,对采用单深度帧内模式编码的编码单元(CU)根据 相邻块重建像素构建像素候选列表,并比较候选列表中的两个像素值;然后,为增加算法的 安全性、保证 视频的质量,在像素值相等和不相等的情况下分别根据密钥和后续块的预测模式范围选择嵌 入块;最后, 根据隐秘信息对嵌入块候选列表所选像素的索引值进行LSB调制。实验结果表明,本文算 法平均每帧嵌入容 量为244bit,编码重建深度视频绘制视点质量的峰值 信噪比(PSNR)值和结构相似度(SSIM)值仅分别平均下降1.41×10－3 dB和6×10－6,码率平均增长0.12%,可见本文算法对3D视频的主观感知质量及码率影响很小。     

Early merge mode decision for texture coding in 3D-HEVC

As the upcoming 3D video coding standard, high efficiency video coding (HEVC) based 3D video coding (3D-HEVC) has been drafted. In 3D-HEVC, the computational complexity of mode decision process is significantly high due to exhaustive modes’ checks for coding units (CU) derived from recursive quad-tree partitioning. In this paper, we propose an early merge mode decision method for complexity reduction of dependent texture views. First, inter-view correlation and hierarchical depth correlation of coding modes are separately analyzed for B frame and P frame. Then, conditions to early determine merge mode coded CUs are derived based on the correlations. All of the early determined CUs only check merge modes in the mode decision process, which brings considerable complexity reduction. Experimental results demonstrate that the proposed method can achieve average 20.4% of encoding time saving for dependent texture views with negligible rate distortion performance loss.     

Fast algorithm based on the sole- and multi-depth texture measurements for HEVC intra coding

In High Efficiency Video Coding (HEVC), intra coding plays an important role, but also involves huge computational complexity due to a flexible coding unit (CU) structure and a large number of prediction modes. This paper presents a fast algorithm based on the sole- and multi-depth texture measurements to reduce the complexity from CU size and prediction mode decisions. For the CU size decision, evaluation results in the CU coding with one and multiple depths are utilized to classify CUs into heterogeneous, homogeneous, depth-prominent and other ones. Fast CU size decisions are made for different kinds of CUs. For the prediction mode decision, the tendencies for different CU sizes are detected based on multiple depths. The number of searching modes is decreased adaptively for the CU size with fewer tendencies. Experimental results show the proposed algorithm by off-line training reduces 53.32% computational complexity, with 1.47% bit-rate increasing.     

Fast intra mode decision for depth coding in 3D-HEVC

The emergent 3D High Efficiency Video Coding (3D-HEVC) is an extension of the High Efficiency Video Coding (HEVC) standard for the compression of the multi-view texture videos plus depth maps format. Since depth maps have different statistical properties compared to texture video, various new intra tools have been added to 3D-HEVC depth coding. In current 3D-HEVC, new intra tools are utilized together with the conventional HEVC intra prediction modes for depth coding. This technique achieves the highest possible coding efficiency, but leads to an extremely high computational complexity which limits 3D-HEVC from practical applications. In this paper, we propose a fast intra mode decision algorithm for depth coding in 3D-HEVC. The basic idea of the proposed algorithm is to utilize the depth map characteristics to predict the current depth prediction mode and skip some specific depth intra modes rarely used in 3D-HEVC depth coding. Based on this analysis, two fast intra mode decision strategies are proposed including reduction of the number of conventional intra prediction modes, and simplification of depth modeling modes (DMMs). Experimental results demonstrate that the proposed algorithm can save 30 % coding runtime on average while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.     

Fast depth map mode decision based on depth–texture correlation and edge classification for 3D-HEVC

The 3D extension of High Efficiency Video Coding (3D-HEVC) has been adopted as the emerging 3D video coding standard to support the multi-view video plus depth map (MVD) compression. In the joint model of 3D-HEVC design, the exhaustive mode decision is required to be checked all the possible prediction modes and coding levels to find the one with least rate distortion cost in depth map coding. Furthermore, new coding tools (such as depth-modeling mode (DMM) and segment-wise depth coding (SDC)) are exploited for the characteristics of depth map to improve the coding efficiency. These achieve the highest possible coding efficiency to code depth map, but also bring a significant computational complexity which limits 3D-HEVC from real-time applications. In this paper, we propose a fast depth map mode decision algorithm for 3D-HEVC by jointly using the correlation of depth map-texture video and the edge information of depth map. Since the depth map and texture video represent the same scene at the same time instant (they have the same motion characteristics), it is not efficient to use all the prediction modes and coding levels in depth map coding. Therefore, we can skip some specific prediction modes and depth coding levels rarely used in corresponding texture video. Meanwhile, the depth map is mainly characterized by sharp object edges and large areas of nearly constant regions. By fully exploiting these characteristics, we can skip some prediction modes which are rarely used in homogeneity regions based on the edge classification. Experimental results show that the proposed algorithm achieves considerable encoding time saving while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.     

Fast encoder decision for texture coding in 3D-HEVC

As a 3D extension of the High Efficiency Video Coding (HEVC) standard, 3D-HEVC is developed to improve the coding efficiency of multi-view video. However, the improvement of the coding efficiency is obtained at the expense of a computational complexity increase. How to relieve the computational burden of the encoder is becoming a critical problem in applications. In this paper, a fast encoder decision algorithm to encode the dependent texture views is proposed, where two strategies to accelerate encoder decision by exploiting inter-view correlations are utilized. The first one is an early merge mode decision algorithm, and the second one is an early CU splitting termination algorithm. Experimental results show that the proposed algorithm can achieve 47.1% encoding time saving with overall 0.1% BD-rate reduction compared to HTM (3D-HEVC test model) version 7 under the common test condition (CTC). Both of the two strategies have been adopted into the 3D-HEVC reference software and enabled as a default encoding process under CTC.     

Self-learning residual model for fast intra CU size decision in 3D-HEVC

As an extension of the High Efficiency Video Coding (HEVC) standard, 3D-HEVC requires to encode multiple texture views and depth maps, which inherits the same quad-tree coding structure as HEVC. Due to the distinct properties of texture views and depth maps, existing fast intra prediction approaches were presented for the coding of texture views and depth maps, respectively. To further reduce the coding complexity of 3D-HEVC, a self-learning residual model-based fast coding unit (CU) size decision approach is proposed for the intra coding of both texture views and depth maps. Residual signal, which is defined as the difference between the original luminance pixel and the optimal prediction luminance pixel, is firstly extracted from each CU. Since residue signal is strongly correlated with the optimal CU partition, it is used as the feature of each CU. Then, a self-learning residual model is established by intra feature learning, which iteratively learns the features of the previously encoded coding tree unit (CTU) generated by itself. Finally, a binary classifier is developed with the self-learning residual model to early terminate CU size decision of both texture views and depth maps. Experimental results show the proposed fast intra CU size decision approach achieves 33.3% and 49.3% encoding time reduction on average for texture views and depth maps with negligible loss of overall video quality, respectively.     

Low-Complexity Intra-Coding Scheme for HEVC

Compared with H.264/AVC, the latest video standard High Efficiency Video Coding (HEVC) known as H.265 improves the coding efficiency by adopting the quadtree splitting structure which is flexible in representing various textural and structural information in images. However, the computational complexity is dramatically increased, especially in the intra-mode decision process owing to supporting more partitions and modes. In this paper, we propose a low-complexity algorithm for HEVC intra-coding, which consists of a fast coding unit (CU) size decision (FCUSD) method and a fast prediction unit (PU) mode decision (FPUMD) method. In FCUSD, unnecessary CU sizes are skipped early according to the depth level of neighboring CUs and the rate distortion (RD) cost threshold derived from the former coded frame. In FPUMD, the PU mode and RD cost correlations between different depth levels are utilized to terminate unnecessary candidate modes. Experimental results demonstrate that the proposed algorithm can achieve about 50.99 % computational complexity reduction on average with 1.18 % BD-rate increase and 0.08 dB BD-psnr loss.     

Low complexity inter coding scheme for Versatile Video Coding (VVC)

The latest coding standard Versatile Video Coding (VVC) developed by the Joint Video Experts Team (JVET) and Video Coding Experts Group (VCEG) was finalized in 2020. By introducing several new coding techniques, VVC improves the compression efficiency by 50% compared with H.265/HEVC. However, the coding complexity increases dramatically, which obstructs it from real-time application. To tackle this issue, a fast inter coding algorithm utilizing coding information is proposed to speed up the coding process. First, by analyzing the coding areas of the neighboring CUs, we predict the coding area of the current CU to terminate unnecessary splitting modes. Then, the temporally optimal coding mode generated during the prediction process is further utilized to shrink the candidate modes to speed up the coding process. Finally, the distribution of neighboring prediction modes are exploited to measure the motion complexity of the current CU, based on which the unnecessary prediction modes can be skipped earlier. Experimental results demonstrate that the proposed method can reduce the coding complexity by 40.08% on average with 0.07 dB BDPSNR decrease and 1.56% BDBR increase, which outperforms the state-of-the-art approach.