An integrated approach to summarization and adaptation using H.264/MPEG-4 SVC

The huge amount of multimedia content and the variety of terminals and networks make video summarization and video adaptation two key technologies to provide effective access and browsing. With scalable video coding, the adaptation of video to heterogeneous terminals and networks can be efficiently achieved using together a layered coding hierarchy and bitstream extraction. On the other hand, many video summarization techniques can be seen as a special case of structural adaptation. This paper describes how some of them can be modified and included in the adaptation framework of the scalable extension of H.264/AVC. The advantage of this approach is that summarization and adaptation are integrated into the same efficient framework. The utility of this approach is demonstrated with experimental results for the generation of storyboards and video skims, showing that the proposed framework can generate the adapted bitstream of the summary faster than a conventional transcoding approach.     

H.264/AVC在视频监控系统中的应用

H.264/AVC是由国际电信联盟(ITU)和国际标准化组织(ISO)共同制定的新一代视频编码标准,首先指出目前采用其他标准设计视频监控系统的不足,介绍和分析H.264/AVC采用的新编码技术,最后根据视频监控系统的特点,主要讨论H.264/AVC在视频监控应用中涉及的关键技术,包括SP/SI帧技术和分层编码技术。     

H.264/AVC在3G移动通信中的应用

H.264/AVC是目前最新,也是性能最优异的国际视频压缩编码标准。在相同的视频质量下,H.264/AVC可以比MPEG-4(SP)节省大约一半的带宽,同时还具有更好的网络适应性和传输健壮性,在3G移动通信系统带宽资源紧张、通信环境恶劣的情况下,H.264/AVC应该是目前最合适的选择。从压缩效率、网络适应性和健壮性3个方面分析了H.264/AVC的新技术,并讨论了它们在3G移动通信中的应用。     

Algorithm analysis and architecture design for HDTV applications - a look at the H.264/AVC video compressor system

In this article, we suggest some techniques to design the H.264/AVC video coding system for HDTV applications. The design exploration is made according to software profiling. The design considerations of system scheduling and pipelining are discussed followed by the architecture optimization of the significant modules. The efficient H.264/AVC video coding system is achieved by combining these techniques.     

Requantization transcoding for H.264/AVC video coding

In this paper, efficient solutions for requantization transcoding in H.264/AVC are presented. By requantizing residual coefficients in the bitstream, different error components can appear in the transcoded video stream. Firstly, a requantization error is present due to successive quantization in encoder and transcoder. In addition to the requantization error, the loss of information caused by coarser quantization will propagate due to dependencies in the bitstream. Because of the use of intra prediction and motion-compensated prediction in H.264/AVC, both spatial and temporal drift propagation arise in transcoded H.264/AVC video streams. The spatial drift in intra-predicted blocks results from mismatches in the surrounding prediction pixels as a consequence of requantization. In this paper, both spatial and temporal drift components are analyzed. As is shown, spatial drift has a determining impact on the visual quality of transcoded video streams in H.264/AVC. In particular, this type of drift results in serious distortion and disturbing artifacts in the transcoded video stream. In order to avoid the spatially propagating distortion, we introduce transcoding architectures based on spatial compensation techniques. By combining the individual temporal and spatial compensation approaches and applying different techniques based on the picture and/or macroblock type, overall architectures are obtained that provide a trade-off between computational complexity and rate-distortion performance. The complexity of the presented architectures is significantly reduced when compared to cascaded decoder–encoder solutions, which are typically used for H.264/AVC transcoding. The reduction in complexity is particularly large for the solution which uses spatial compensation only. When compared to traditional solutions without spatial compensation, both visual and objective quality results are highly improved.     

视频编码新标准H.264/AVC中的重要技术

简要介绍了最新的视频编码标准H．264／AVC的制定情况，详细叙述了H．264／AVC基本框架中采用的几个重要技术，最后比较了H．264／AVC和已有的几个编码标准的性能，H．264／AVC的压缩性能最好。     

Optimal transmission of high definition video transmission in WiMedia systems

WiMedia systems are developed for indoor high-data-rate wireless systems. The H.264/AVC, a high-efficiency video coding technique, is considered for high definition (HD) video application. Considering the transmission of H.264/AVC based HD video over WiMedia, in this paper, we will propose a cross-layer architecture and an analytical model to calculate the optimal payload length with the constraints in error criteria, retransmission mechanism, and the delay budget. Besides, the required minimum reservation slots in WiMedia are also investigated to optimize the transmission performance of HD video.     

Efficient error resilient algorithm for H.264/AVC: mobility management in wireless video streaming

The H.264/AVC standard introduces enhanced error robustness capabilities enabling resilient and reliable transmission of compressed video signals over wireless lossy packet networks. Those robustness capabilities are achieved by integrating some new error resilience tools that are essential for a proper delivery of real-time video services. Those tools include the Intra Refreshing (IR), Arbitrary Slice Ordering (ASO), Sequence Picture Parameter Sets (PPS), Redundant Slices (RS) tools and Flexible Macroblock Ordering (FMO). This paper presents an error resilient algorithm in wireless H.264/AVC streaming. The proposed method merges Reference Frame Selection (RFS), Intra Redundancy Slice and Adaptive Intra Refreshment techniques in order to prevent temporal error propagation in error-phone wireless video streaming. The coding standards only specify the decoding process and the bitstream syntax to allow considerable flexibility for the designers to optimize the encoder for coding performance improvement and complexity reduction. Performance evaluations demonstrate that the proposed encoding algorithm outperforms the conventional H.264/AVC standard. Both subjective and objective visual quality comparative study has been also carried out in order to validate the proposed approach. The proposed method can be used and integrated into H264/AVC without violating the standard.     

Mode-Dependent Templates and Scan Order for H.264/AVC-Based Intra Lossless Coding

In H.264/advanced video coding (AVC), lossless coding and lossy coding share the same entropy coding module. However, the entropy coders in the H.264/AVC standard were original designed for lossy video coding and do not yield adequate performance for lossless video coding. In this paper, we analyze the problem with the current lossless coding scheme and propose a mode-dependent template (MD-template) based method for intra lossless coding. By exploring the statistical redundancy of the prediction residual in the H.264/AVC intra prediction modes, more zero coefficients are generated. By designing a new scan order for each MD-template, the scanned coefficients sequence fits the H.264/AVC entropy coders better. A fast implementation algorithm is also designed. With little computation increase, experimental results confirm that the proposed fast algorithm achieves about 7.2% bit saving compared with the current H.264/AVC fidelity range extensions high profile.     

Efficient Level and Zero Coding Methods for H.264/AVC Lossless Intra Coding

Since H.264/AVC was designed mainly for lossy video coding, the entropy coding methods in H.264/AVC are not appropriate for lossless video coding. Based on statistical differences of residual data in lossy and lossless coding, we develop efficient level and zero coding methods. Therefore, we design an improved context-based adaptive variable length coding (CAVLC) scheme for lossless intra coding by modifying the relative entropy coding parts in H.264/AVC. Experimental results show that the proposed method provides approximately 6.8% bit saving, compared with the H.264/AVC FRExt high profile.      

Motion Estimation Based on Spline Interpolation in H.264/AVC Video Coder for Videoconferencing Application: Performance Versus Computation Load

The powerful H.264/AVC video coder involves a large encoding computational cost than the existing video standards due mainly to the motion-compensated estimation scheme based on a full search of multiple reference frames in the sequence. This strategy decreases the residual errors of the predicted frames and may improve the performance of the video coder. However a great number of computations are usually wasted without improving significantly the quality of the decoded video mostly in videoconferencing applications. To reduce the encoding computational load and preserve the performance of the video coder, this paper proposes to substitute the motion-compensated estimation method implemented in H.264/AVC by a temporal spline interpolation. Simulations on several test sequences show that important encoding saving times are achieved with a competitive quality of the decoded video compared to the exhaustive search of multiple reference frames in the H.264/AVC video coder.     

新一代运动图像压缩标准H.264/AVC

H.264/AVC是ISO/IECMPEG与ITU-TVCEG共同制订的新一代图像编码标准,它具有更高的压缩性能。文章介绍H.264/AVC图像编解码系统的实现过程,并对其采用的新技术进行描述。     

Efficient rate control scheme for low bit rate H.264/AVC video coding

This article presents an efficient rate control scheme for H.264/AVC video coding in low bit rate environment. In the proposed scheme, an improved rate-distortion (RD) model by both analytical and empirical approaches is developed. It involves an enhanced mean absolute difference estimating method and a more rate-robust distortion model. Based on this RD model, an efficient macroblock-layer rate control scheme for H.264/AVC video coding is proposed. Experimental results show that this model encodes video sequences with higher peak signal-to-noise ratio gains and generates bit stream closer to the target rate.     

结合空时域下采样与重建的H.264/AVC压缩性能优化

为提高H.264/AVC标准在带宽资源严重受限时的压缩效率,采用空时域相结合的编码思路,提出了一种基于运动检测的自适应抽帧方法,并结合空域下采样与重建研究了一种改进的H.264/AVC压缩性能优化框架。在编码端,原视频先空域下采样以减少空间分辨率,然后根据视频运动特征,采用不同抽帧模式自适应地降低帧率,再经H.264/AVC编码,有效降低了编码码率。在解码端,解码视频则采用与抽帧模式相对应的运动估计与补偿插帧方法重建出抽取帧,再利用超分辨率重建技术将视频恢复到原空间分辨率。实验结果表明,所提方法在低码率段的视频压缩性能优于H.264/AVC标准编解码及相关文献方法。     

AND-OR tree-based mode selection for video coding

The conventional video coding approach is pragmatic in that researchers design different methods and compare their results to state-of-the-art methods. Although the H.264/AVC baseline is effective, there is a need to develop an abstract view of video coding, in the hope that new insights can be derived from the abstraction. In this paper, we propose a preliminary approach based on an AND-OR tree representation of video coding. We show that the H.264/AVC baseline can be represented as an AND-OR tree structure. Based on the AND-OR tree representation, we propose two video coding systems: one is a T+2D wavelet codec based on a motion-compensated temporal filtering (MCTF) lifting structure, and the other is the AND-OR tree implementation of the H.264/AVC baseline. We also compare the proposed systems’ coding performance in terms of the PSNR with that of H.264/AVC JM 16.2.     

H.264/AVC视频编码标准的技术特点及应用分析

H.264／AVC是ITU-T视频编码专家组和ISO／IEC运动图像专家组(MPEG)共同开发出来的最新视频编码标准,它具备许多新的技术特点,如:多模式运动预测、柔性宏块排序、整数变换、一致性变字长编码、内容自适应二进制算术编码、分层编码、错误约束机制、错误掩盖技术及高效的比特流切换技术等等。本文在阐述该标准系统层结构的基础上,分析其主要技术特点及H.264／AVC VBR码流特性,阐述该标准在IP网络及无线网络中的应用情况。     

一种面向H.264/AVC的码率控制算法

码率控制是视频编码中非常重要的技术之一,任何标准离开码率控制其应用都会受到限制.H.264/AVC是目前最新的视频编码标准,本文根据H.264/AVC编码标准的特性及其HRD部分对码率控制的要求,提出了一种新的适合H.264/AVC的码率控制算法,该算法实现了率失真优化与码率控制的结合,使得在达到码率控制的同时也能保证较高的编码效率,同时在码率控制的过程中根据HRD缓冲区状态进行位分配调整,保证了编解码缓冲区既不上溢又不下溢.该算法作为技术提案已被H.264/AVC接受,并集成到H.264/AVC的校验模型软件中.     

Fast Interframe mode decision algorithm based on mode mapping and MB activity for MPEG-2 to H.264/AVC transcoding

Recently the latest video coding standard H.264/AVC is widely used for the mobile and low bitrate video codec in the various multimedia terminals. On the other hand, the MPEG-2 MP@HL codec has become the center of digital video contents since it is the standard codec for the Digital TV (DTV). To provide the bridge between the contents in MPEG-2 and mobile terminals, the transcoding of MPEG-2 contents into H.264/AVC format is an inevitable technology in the digital video market. The main bottleneck in the process lies in the computational complexity. In H.264/AVC, the variable block size (VBS) mode decision (MD) is used in the Interframe for the improved performance in the motion compensated prediction. For the macroblock (MB) which cannot be accurately predicted with one motion vector (MV), it is partitioned into smaller blocks and predicted with different MVs. In addition, SKIP and Intra modes are also permitted in the Interframe MD of H.264/AVC to further ameliorate the encoding performance. With the VBS MD technology, the Inter prediction accuracy can be improved significantly. However, the incidental side-effect is the high computational complexity. In this paper, we propose a fast Interframe MD algorithm for MPEG-2 to H.264/AVC transcoding. The relationships between SKIP and Intra modes are detected at first to map these two kinds of modes directly from MPEG-2 to H.264/AVC. And then the MB activity will be scaled by the residual DCT energy obtained from the MPEG-2 decoding process to estimate the block sizes of the MB mode for H.264/AVC Interframe MD. In our proposed method, the original redundant candidate modes can be eliminated effectively, resulting in the reduction of the computational complexity. It can reduce about 85% Rate-to-Distortion Cost (RDCost) computing and 45% entire processing time compared with the well-known cascaded transcoder while maintaining the video quality.     

A generic,comprehensive and granular decoder complexity model for the H.264/AVC standard

With recent advances in computing and communication technologies, ubiquitous access to high quality multimedia content such as high definition video using smartphones, netbooks, or tablets is a fact of our daily life. However, power consumption is still a major concern for portable devices. One approach to address this concern is to control and optimize power consumption using a power model for each multimedia application, such as a video decoder. In this paper, a generic, comprehensive and granular decoder complexity model for the baseline profile of H.264/AVC decoder has been proposed. The modeling methodology was designed to ensure a platform and implementation independent complexity model. Simulation results indicate that the proposed model estimates decoder complexity with an average accuracy of 92.15% for a wide range of test sequences using both the JM reference software and the x264 software implementation of H.264/AVC, and 89.61% for a dedicated hardware implementation of the motion compensation module. It should be noted that in addition to power consumption control, the proposed model can be used for designing a receiver-aware H.264/AVC encoder, where the complexity constraints of the receiver side are taken into account during compression. To further evaluate the proposed model, a receiver-aware encoder has been designed and implemented. Our simulation results indicate that using the proposed model the designed receiver aware encoder performs similar to the original encoder, while still being able to satisfy the complexity constraints of various decoders.     

Fast multi-frame motion estimation for H264/AVC system

Advanced video compression standard, H264/AVC, with multi-frame motion estimation, can offer better motion-compensation than the previous coding standards. However, the implementation of real-time multi-frame estimation for an H264/AVC system is difficult due to heavy computations. In this paper, a fast algorithm is proposed in an effort to reduce the searching computation for motion estimation with five reference frames. The fast multi-frame motion estimation consists of the adaptive full-search, three-step search, and diamond search methods using the content adaptive control process. Efficient control flow is proposed to select the searching algorithm dependent on video features. The adaptive algorithm can achieve better rate-distortion and lower computation for H264/AVC coding. The experiments indicate that the speed-up is 6–15 times compared with the full search method, while the image quality slightly degrades.