H.264/AVC中基于决策树的P帧快速模式选择（HHME2013第61号论文）

目的：H.264/AVC帧间预测编码需要对所有可能编码模式计算并比较率失真代价，众多的模式类型导致了P帧编码的计算复杂度非常高。本文提出了一种针对P帧的基于决策树的快速选择候选模式算法。方法：在对宏块进行16×16的帧间运动估计后，首先根据残差宏块中4×4全零系数块个数对部分宏块直接选择出候选模式；然后使用16个4×4块的SATD值，采用决策树分类方法对其余宏块选择候选模式。结果：由于只需对候选模式进行编码，因此有效降低了编码器的计算复杂度。实验结果表明，与原始全搜索编码算法相比，该算法对不同运动程度的视频序列获得了较一致的编码时间节省，同时平均峰值信噪比的损失和平均比特率的增加均较少。结论：本文提出了一种新的P帧帧间预测候选模式选择算法，根据帧间运动估计后的残差宏块信息，采用决策树方法对候选模式集进行分类。实验结果表明，该算法能在保证视频编码质量的前提下，有效地降低编码过程中的计算量，缩短编码时间。

A fast intermode decision based on the decision tree for H.264/AVC P-frame encoding

Objective: H.264/AVC encoding standard effectively improves coding efficiency by using the intra and inter prediction techniques. However, due to the use of variable block sizes and rate-distortion optimization techniques, there is extremely high computational complexity for inter prediction in H.264/AVC which limits the applications of H.264/AVC. This is because the encoder needs to exhaustively check all the prediction modes for identifying the best one. Therefore, reducing the complexity of the encoder is very important especially for real-time applications. Choosing a small number of candidate modes other than all the modes to perform rate-distortion optimization can reduce the complexity and speed up the encoding process. In this paper, a fast intermode decision algorithm for P-frame encoding based on the decision tree is proposed to reducing the number of modes required to be checked. Method: The inter frame motion estimation for 16x16 size is firstly performed when a macroblock (MB) of P-frame is encoded by using the inter prediction technique. And we use the information of residual MB after motion compensation to select a small number of candidate modes from all possible prediction modes to calculate and compare the rate-distortion cost. At first, the number of 4x4 all-zero blocks in the residual MB can describe how accurate the 16x16 inter prediction is. Thus with the statistical analysis of the correlation between the number of 4x4 all-zero blocks in the residual MB and the best prediction mode in full search algorithm, the candidate modes of some MBs can be directly determined based on the number of all-zero blocks in the residual MB. Then for the remaining MBs, machine learning tools are used to exploit the correlation between the residual MB information and the best mode in H.264/AVC full search algorithm. By using the decision tree classification algorithm, candidate modes can be selected based on the SATD features of residual MB. Results: The proposed algorithm efficiently reduces the number of candidate modes for P-frame inter prediction in H.264/AVC encoder. Experimental results show that the proposed algorithm can achieve the significant encoding time saving for all the test video sequences which covering a wide range of motion activities, and meanwhile there is the less PSNR degradation and a little bit rate increase, comparing with the reference full search algorithm. Although the time saving of the proposed algorithm is not better than those of the comparative algorithms, the time savings for video sequences with different motion degrees are almost same and the rate distortion performance is obvious better. Conclusions: H.264/AVC video coding standard effectively improves coding efficiency but at the cost of high computational complexity. This paper presents a fast and efficient intermode decision algorithm to speed up the encoding process. The algorithm reduces the computational complexity by selecting a small number of candidate modes from the set of all possible prediction modes. We use the decision tree classification algorithm to determine candidate modes based on the information in residual MB after 16x16 inter motion estimation. The experimental results indicate that our algorithm reduces the computational complexity with a little loss in PSNR and increment in the total bit rate.