An improved hybrid fast mode decision method for H.264/AVC intra coding with local information

In this paper, an improved hybrid fast mode decision method for H.264/AVC intra coding is proposed, which is based on the analysis of edge filter and more efficient mode selection. In contrast to the conventional fast mode decision methods where multiple neighbor modes are determined by the same filter, a unique filter is applied to each directional mode to eliminate the correlation among neighbor modes. To reduce the complexity, the filters are only applied to the selected pixels that cover most of the block region. After that, the results along prediction residuals of these pixels are summed for fast mode decision. Moreover, local information is also taken into account through Most Probable Mode (MPM) from neighbor-coded blocks. Experimental results demonstrated that better coding performance is achieved as compared with well-known conventional fast mode decision algorithms for H.264/AVC intra coding.