Efficient multi-view video coding using inter-view information

Multi-view video coding (MVC) has been extended from H.264/AVC to improve the coding efficiency of multi-view video. This paper proposes a fast mode decision algorithm which can make an early decision on the correct mode partition to solve the issue of the enormous computational complexity. The best modes of the reference views are utilized to determine the complexity of the macroblock (MB) in the current view, the mode candidates needed to be calculated can then be obtained according to the complexity. If the complexity is low or medium, the search range can be reduced. The threshold of the rate-distortion cost for the current MB is calculated using the co-located and neighboring MBs in previously coded view and is utilized as the criterion for early termination. The motion vector difference in the reference view is applied to dynamically adjust the search range in the current MB. Experimental results prove that the proposed algorithm achieves a time saving of 81.05% for a fast TZ search and 87.85% for full search, and still maintains quality performance and bitrate.     

Fast macroblock encoding algorithm based on rate-distortion activity for multiview video coding

Multiview video coding (MVC) is the appendix H of H.264/AVC, and it requires a great amount of time to compress multiple viewpoints׳ video with complex prediction structures. To reduce the whole computational complexity of MVC, this paper proposes a fast macroblock (MB) encoding algorithm based on rate-distortion (RD) activity, and it includes the fast mode decision and the fast motion/disparity estimation. First, the RD activity type of the current MB is calculated by utilizing the Skip/Direct RD cost and the average RD costs of classified MB modes. Then, through utilizing the RD activity type and RD costs of the estimated modes, the selection of candidate modes, the early decision of Skip/Direct mode, and the reduction of Inter8×8 mode estimation are all presented in the fast mode decision. By using the RD activity type and the correlations of vectors, the selection of search center and the prediction of search range are introduced in the fast motion/disparity estimation. In addition, the proposed algorithm can be applied to temporal and inter-view views as well as anchor and non-anchor frames. An experiment with a wide range of video scenes, camera setups and quantization parameters was implemented, and the results confirmed that the proposed algorithm can reduce the encoding time significantly while maintaining a similar RD performance as the original MVC encoder. Compared to the state-of-the-art algorithms, the proposed algorithm also demonstrated better performances in the various test cases.     

Fast depth map mode decision based on depth–texture correlation and edge classification for 3D-HEVC

The 3D extension of High Efficiency Video Coding (3D-HEVC) has been adopted as the emerging 3D video coding standard to support the multi-view video plus depth map (MVD) compression. In the joint model of 3D-HEVC design, the exhaustive mode decision is required to be checked all the possible prediction modes and coding levels to find the one with least rate distortion cost in depth map coding. Furthermore, new coding tools (such as depth-modeling mode (DMM) and segment-wise depth coding (SDC)) are exploited for the characteristics of depth map to improve the coding efficiency. These achieve the highest possible coding efficiency to code depth map, but also bring a significant computational complexity which limits 3D-HEVC from real-time applications. In this paper, we propose a fast depth map mode decision algorithm for 3D-HEVC by jointly using the correlation of depth map-texture video and the edge information of depth map. Since the depth map and texture video represent the same scene at the same time instant (they have the same motion characteristics), it is not efficient to use all the prediction modes and coding levels in depth map coding. Therefore, we can skip some specific prediction modes and depth coding levels rarely used in corresponding texture video. Meanwhile, the depth map is mainly characterized by sharp object edges and large areas of nearly constant regions. By fully exploiting these characteristics, we can skip some prediction modes which are rarely used in homogeneity regions based on the edge classification. Experimental results show that the proposed algorithm achieves considerable encoding time saving while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.     

基于深度图像的H.264快速帧间模式选择算法

H.264编码标准的帧间模式选择算法采用全候选模式搜寻算法,为编码器带来了高运算复杂度。提出一种快速帧间模式选择算法,旨在降低H.264帧间编码的运算复杂度。算法利用深度图像信息从彩色图像中分割出背景区域和运动剧烈的区域,宏块根据所在的区域选择相应的编码策略,从而达到缩小候选模式范围,加快模式选择速度的目的。实验证明,该算法在图像质量基本不变的情况下编码时间平均节省23%以上。     

利用纹理结构的HEVC快速帧内模式选择算法

高性能视频编码（HEVC）是刚确立的最新一代的视频编码标准。对于帧内编码,HEVC最大的特点是采用了从64×64至8×8的编码单元划分和35种帧内预测模式,HEVC通过遍历所有的分块和帧内预测模式,选取最优的预测方式,这种帧内预测方式在提高预测精度的同时也大大增加了编码的计算复杂度。为了降低帧内预测的计算复杂度,HM2.0版以上的HEVC测试软件采用一种基于两步预测的快速帧内模式选择算法。在此基础上本文首先提出了一种基于纹理方向的快速粗选方案,减少参与计算的粗选模式数,进而提出基于结构相关的决策方法,利用同质图像区域的纹理相似性,减少参与率失真代价函数计算的候选模式数量,进一步降低了帧内预测计算复杂度。实验结果表明,本文所提出的快速算法,在保持编码质量基本不变的条件下,可以使基于两步预测的快速帧内模式选择时间缩短25%。      

Efficient Mode Decision Algorithm Based on Spatial,Temporal, and Inter‐layer Rate‐Distortion Correlation Coefficients for Scalable Video Coding

The layered coding structure of scalable video coding (SVC) with adaptive inter‐layer prediction causes noticeable computational complexity increments when compared to existing video coding standards. To lighten the computational complexity of SVC, we present a fast algorithm to speed up the inter‐mode decision process. The proposed algorithm terminates inter‐mode decision early in the enhancement layers by estimating the rate‐distortion (RD) cost from the macroblocks of the base layer and the enhancement layer in temporal, spatial, and inter‐layer directions. Moreover, a search range decision algorithm is also proposed in this paper to further increase the motion estimation speed by using the motion vector information from temporal, spatial, or inter‐layer domains. Simulation results show that the proposed algorithm can determine the best mode and provide more efficient total coding time saving with very slight RD performance degradation for spatial and quality scalabilities.     

A novel fast intra mode decision for versatile video coding

The latest video coding standard Versatile Video Coding (VVC) obtains superior coding efficiency compared to the High Efficiency Video Coding (HEVC), which is achieved by incorporating more effective and complex new coding tools. In this paper, we propose a novel fast intra mode decision algorithm for VVC, including following two strategies: (1) the correlation between the optimal modes of the adjacent blocks and the modes selected in the rough modes decision (RMD) process is analyzed and applied to reduce the modes in the candidate list; (2) modes in the candidate list are sorted in ascending order according to the modes’ cost calculated in the RMD process. An early termination method is proposed for terminating the optimal prediction mode decision process based on this new order early. These two strategies are incorporated into intra coding to reduce the coding complexity. Since these two strategies do not add any additional computational complexity, the proposed fast algorithm can achieve more complexity reduction. The experimental results show that the complexity reduction of the proposed algorithm is up to 44.74% compared to VVC reference software VTM2.0, and averagely 30.59% encoding time saving with 0.86% BDBR increase.     

应用于多视点视频压缩的多参考B帧快速编码算法

基于H.264平台就多视点视频压缩中多参考帧技术存在计算复杂度高的现状,根据多视点视频序列特性,提出了一种针对多参考B帧的快速多视点视频编码压缩算法。新算法改进了多参考帧列表策略,并结合多视点视频序列的时空域相关性特性,提出了适用于多视点视频编码的参考帧列表调整策略。此外,还针对多种块模式编码引入了块模式预判策略。新算法在保证率失真性能的基础上,有效地降低了计算复杂度,提高了多视点视频编码速度。虽然本文是基于M-Picture编码结构实现了新算法,但它同样适用于其它应用多参考B帧预测技术的多视点视频编码结构中。     

采用率失真与模式特征的多视点视频编码快速模式选择

现有的多视点视频编码使用了分层B帧（Hierarchical B Picture, HBP）的预测结构,其帧内预测、帧间预测以及视点间预测的模式选择给多视点视频编码带来了庞大的计算复杂度。针对这一问题,我们在分析了JMVC模式分布比例的基础上,提出了一个快速帧间模式选择的算法。这种算法利用率失真代价和预测模式特征之间的关系来及时判定最优模式:如果上一尺寸预测模式的率失真代价小于当前尺寸预测模式的率失真代价则认为上一预测模式为最优模式,跳过检查其他更小尺寸的预测模式;反之,如果上一尺寸的预测模式的率失真代价大于当前尺寸的预测模式的率失真代价,则继续检查其他更小的尺寸。这样,通过提前终止一些不必要的模式选择过程,多视点视频编码的计算量得到大幅的降低。实验结果表明:所提算法能在保持JMVC中全搜索算法的编码效率同时,使计算复杂度减少了81.66%。      

低复杂度的多视点视频编码宏块模式决策算法

为了降低多视点视频编码(MVC)的计算复杂度,提出了一种基于全局-局部率失真代价的低复杂度宏块模式决策算法。首先根据宏块候选模式的块尺寸和率失真代价特点,将所有候选模式分为大尺寸模式(Skip/Direct、Inter16×16和Intra16×16)和小尺寸模式(Inter16×8、Inter8×16、Inter8×8、Intra8×8和Intra4×4);接着统计已编码帧中这两类宏块模式的平均率失真代价,并利用这些平均率失真代价计算当前帧宏块模式决策的全局率失真代价;最后利用得到的全局率失真代价,并结合当前宏块已估计模式的局部率失真代价和空间邻近宏块的模式信息提前终止模式决策,具体包括对大尺寸宏块模式进行提前判定以及对小尺寸候选模式进行逐级选择。实验结果表明:与MVC参考代码中的全搜索模式决策算法相比,本文算法在时域预测视点和视点域预测视点上分别节省了74%和82%的平均编码时间,同时保持了良好的编码率失真性能;与现有模式决策快速算法相比,本文算法降低了更多的编码时间,并具有更好的编码率失真性能。     

H.264的帧间宏块模式选择算法

H.264中定义了7种不同大小的帧间宏块模式,为了提高编码效率,编码时对每一种宏块模式都进行运动搜索,并利用率失真优化(RDO)分别计算编码代价,选择编码代价最小的模式作为帧间编码模式,但是计算量非常大,导致运动估计速度缓慢,很难实现实时编码。本文提出一种选择宏块模式的快速算法,通过宏块模式编码代价的比较,提前确定宏块模式,从而逐步减少不必要的宏块搜索和RDO计算。实验结果表明,此方法与全宏块模式搜索相比较,图像质量和码流只有少量变化,而编码速度有显著提高。     

Sum-of-gradient based fast intra coding in 3D-HEVC for depth map sequence (SOG-FDIC)

As the latest video coding standard for multi-view plus depth video, 3D-HEVC yields high coding efficiency but at the cost of heavy computational complexity. To reduce the computational complexity, a fast intra coding algorithm based on sum-of-gradient criterion for depth map coding in 3D-HEVC, named SOG-FDIC, is proposed in this paper. Based on the observation that DMM modes and smaller partitioning sizes are rarely used in flat region, sum of gradient is presented to determine whether the current block belongs to the flat region so as to skip unnecessary checking of DMMs and smaller partitioning sizes. Experimental results show that the proposed algorithm can save about 21.8% coding time while keeping almost the same coding efficiency and the reconstructed video quality of depth maps and synthesized views, compared with the original 3D-HEVC. Moreover, it has been verified that the proposed method outperforms the state-of-the-art methods.     

Fast block mode decision algorithm in H.264/AVC video coding

The recent video coding standard H.264/AVC show extremely higher coding efficiency compare to any other previous standards. H.264/AVC can achieve over 50% of bit rate saving with same quality using the rate–distortion process, but it brings high computational complexity. In this paper, we propose an algorithm that can reduce the complexity of the codec by reducing the block mode decision process adaptively. Block mode decision process in H.264/AVC consists of inter mode decision process and intra mode decision process. We deal with reduction method for inter and intra mode decision. In this paper an efficient method is proposed to reduce the inter mode decision complexity using the direct prediction methods based on block correlation and adaptive rate distortion cost threshold for early stopping. The fast intra mode reduction algorithm based on inter mode information is also proposed to reduce the computational complexity. The experimental results show that the proposed algorithm can achieve up to 63.34–77.39% speed up ratio with a little PSNR loss. Increment in bit requirement is also not much noticeable.     

结合深度信息的多视点视频编码快速模式选择算法

多视点视频加深度(MVD,multi-view video plus depth)的编码格式包含多个纹理视频序列及其对应的深度图,深度图与对应的纹理视频具有相似的边缘信息。传统的编码模式不考虑两者的联系,单独编码导致复杂度高、编码时间过长。因此,合理利用深度图与纹理图的相似性进行编码,可以有效降低编码复杂度,同时应该确保编码质量不受影响。本文利用深度图的这一特点辅助纹理视频的编码,提出一种帧间快速模式选择算法,充分利用深度图与纹理图之间的相似性,建立一种联合复杂度模型,根据模型得到每个宏块的复杂度。对于复杂度低的宏块,在计算率失真代价之前跳过编码中一些不必要的模式,从而降低编码复杂度。实验表明,本文提出的快速模式选择算法,在保证率失真性能基本不变的情况下,能减少60.57%的编码复杂度,并最高能减少80.64%的编码时间。     

基于结构张量的H.266帧内预测快速算法

新一代视频编码标准H.266将帧内预测模式从35种扩展到67种,提高预测准确率的同时大大增加了编码的计算复杂度。本文根据视频内容纹理方向和帧内预测模式选择相关性强的特点提出了一种基于结构张量的H.266帧内预测模式快速选择算法,以编码单元CU为单位,计算其结构张量矩阵,再依据结构张量奇异值分解后得到的特征向量来确定当前CU的纹理方向,缩小候选模式的选择范围,减少进行代价计算的模式数量。实验结果表明,在全I帧条件下,该算法相较于H.266 VTM-7.3标准平台在节省17.28%编码时间的同时只增加1.12%的码率,有效地降低了编码复杂度。      

基于时空相关性的HEVC帧间模式决策快速算法

新一代的高效率视频编码标准HEVC采用编码树单元（CTU）四叉树划分技术和多达10种的帧间预测单元（PU）模式,有效地提高了编码压缩效率,但也极大地增加了编码计算复杂度。为了减少编码单元（CU）的划分次数和候选帧间PU模式个数,提出了一种基于时空相关性的帧间模式决策快速算法。首先,利用当前CTU与参考帧中相同位置CTU、当前帧中相邻CTU的深度信息时空相关性,有效预测当前CTU的深度范围。然后,通过分析当前CU与其父CU之间的最佳PU模式空间相关性,以及利用当前CU已估计PU模式的率失真代价,跳过当前CU的冗余帧间PU模式。实验结果表明,提出的算法与HEVC测试模型（HM）相比,在不同编码配置下降低了52%左右的编码时间,同时保持了良好的编码率失真性能;与打开快速算法选项的HM相比,所提算法进一步降低了30%左右的编码时间。     

Low-Cost H.264/AVC Inter Frame Mode Decision Algorithm for Mobile Communication Systems

One of the important issues of green mobile networking is the low energy consumption for either mobile devices or transmissions. To adapt this, a low-cost Inter frame mode decision (MD) algorithm is proposed for H.264/AVC encoder to reduce the computational complexity of the original encoding procedure in this paper. The information extracted from macroblock (MB), such as energy, temporal domain mode similarity and so on, which can be used to pre-estimate the optimal mode of the MB is investigated and utilized to eliminate the redundant mode candidates. The performance evaluations including quantitative analysis and PC simulations show that the proposed algorithm is an energy-efficient source coding because it can reduce around 85% Inter frame encoding time with little quality loss. It can be widely implemented in green mobile networking systems with H.264/AVC standard to realize the real-time video signal coding.     

Fast intra mode decision for depth coding in 3D-HEVC

The emergent 3D High Efficiency Video Coding (3D-HEVC) is an extension of the High Efficiency Video Coding (HEVC) standard for the compression of the multi-view texture videos plus depth maps format. Since depth maps have different statistical properties compared to texture video, various new intra tools have been added to 3D-HEVC depth coding. In current 3D-HEVC, new intra tools are utilized together with the conventional HEVC intra prediction modes for depth coding. This technique achieves the highest possible coding efficiency, but leads to an extremely high computational complexity which limits 3D-HEVC from practical applications. In this paper, we propose a fast intra mode decision algorithm for depth coding in 3D-HEVC. The basic idea of the proposed algorithm is to utilize the depth map characteristics to predict the current depth prediction mode and skip some specific depth intra modes rarely used in 3D-HEVC depth coding. Based on this analysis, two fast intra mode decision strategies are proposed including reduction of the number of conventional intra prediction modes, and simplification of depth modeling modes (DMMs). Experimental results demonstrate that the proposed algorithm can save 30 % coding runtime on average while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.     

一种低复杂度的HEVC帧内快速编码算法

为了降低高效视频编码(HEVC)帧内编码复杂度,提出一种HEVC帧内快速编码算法。根据视频图像的纹理复杂性,提前跳过或者中止部分尺寸的编码单元(CU)的划分,减少CU深度遍历区间;同时,根据粗选过程后预测模式和代价值的统计特性采用阈值法或者梯度模式直方图法进一步筛选掉粗选后可能性较小的预测模式,从而减少最后进行率失真(RD)代价计算的帧内预测模式数量,进一步降低编码复杂度。实验结果表明,本文算法与HEVC原始平台相比,在全I帧编码模式下编码时间平均减少42.20%,码率(BR)上升约1.75%,峰值信噪比(PSNR)降低了0.108dB,有利于实时应用。     

3D-HEVC合并模式快速判决方法研究

基于高效视频编码的3D视频编码（3D-HEVC）是目前正在研究的新一代3D视频编码标准。为降低3D-HEVC中模式选择的计算复杂度,根据非独立视点纹理图中合并模式采用率高的特点,该文提出了一种3D-HEVC合并模式快速判决方法。在B帧中,分析了当前编码单元（CU）与视点方向参考帧中参考块间编码模式的相关性;在P帧中,分析了位于相邻划分深度的CU间编码模式的相关性。根据分析的视点间和划分深度间的相关性设计快速判决条件,预判采用合并/合并-跳过模式编码的CU,判别出的CU在模式选择过程中只检查相关的候选预测模式,从而降低计算复杂度。实验结果表明,与3D-HEVC原始算法相比,该文算法能够在率失真性能损失很小的前提下,平均节省11.2%的总编码时间和25.4%的非独立视点纹理图的编码时间。