Simple intra prediction algorithms for heterogeneous MPEG-2/H.264 video transcoders

Recent developments have given birth to H.264/AVC: a video coding standard offering better bandwidth to video quality ratios than MPEG-2. It is expected that the H.264/AVC will take over the digital video market, replacing the use of MPEG-2 in most digital video applications. The complete migration to the new video-coding algorithm will take several years given the wide scale use of MPEG-2 in the market place today. This creates an important need for MPEG-2/H264 transcoding technologies. However, given the significant differences between both encoding algorithms, the transcoding process of such systems is much more complex to other heterogeneous video transcoding processes. In this work, we start by analyzing the methods defined in the H.264 video coding standard for the intra prediction: a central element of every H.264 encoder. We then introduce and evaluate six fast intra mode decision algorithms which should enable the development of MPEG-2 to H.264 transcoders. Having evaluated all the proposed methods, we have come out with a high-efficient method, namely DC-ABS pixel. Our results show that our algorithm considerable reduces the complexity involved in the intra prediction with respect the mode decision algorithms used in H.264 JM reference software, while exhibiting a slight degradation on the RD function.. Finally, we analyze a comparative study with two of the most prominent fast intra prediction methods presented in the literature. The results show that the proposed DC-ABS pixel method achieves the best results for video transcoding applications.

Effective algorithms for fast transcoding of AVS to H.264/AVC in the spatial domain

This paper proposes a transcoding scheme from AVS to H.264/AVC. As high-compression video coding standards, H.264/AVC jointly developed by MPEG and ITU and AVS developed by the Audio Video Coding Standard Working Group of China will co-exist in the future market. Therefore, it is worthy to transcode the AVS format to the H.264/AVC format or vice versa. After an insight into the inter transcoding from AVS to H.264/AVC, a simple and effective method is proposed by reusing the mode and motion vectors to achieve high-efficient and fast transcoding. The problem in reusing the skip mode is studied and an effective method to eliminate the artifacts is proposed. Furthermore, a fast intra transcoding algorithm based on the distribution of the DCT coefficients is proposed to speed up the transcoding process. Detailed experiment results demonstrate that the proposed algorithm can effectively reduce the transcoding complexity.

A first approach to speeding-up the inter mode selection in MPEG-2/H.264 transcoders using machine learning

The H.264 standard achieves much higher coding efficiency than the MPEG-2 standard, due to its improved inter and intra prediction modes which come with a cost of higher computation complexity. Transcoding MPEG-2 video to H.264 is important to enable gradual migration to H.264. However, given the significant differences between the MPEG-2 and the H.264 coding algorithms, transcoding is much more complex and new approaches to transcoding are necessary. In this paper, we introduce and evaluate a low complexity macroblock partition mode decision algorithm, to be used as part of a high-efficient inter-frame prediction in MPEG-2 to H.264 transcoder. The proposed tools are used to compute an optimal MB coding mode decision with significantly reduced computational complexity. Specifically, we achieve the computational savings by using the following MB information coming from MPEG-2: the MB coding modes, the coded block pattern (CBPC) in MPEG-2, and the mean and variance of the 16 4 × 4 sub blocks of the MPEG-2 residual MBs. We use data mining algorithms to develop a decision tree for H.264 coding mode decisions. The decision trees are built using RD optimized mode decisions and result in highly efficient mode decisions, with significantly reduced computational complexity. The proposed transcoder is 35% faster than the RD optimized H.264 reference transcoder without a significant PSNR degradation (0.05 dB on average). The proposed transcoder performs over 0.4 dB better on average than the SAE cost based H.264 transcoding.

MPEG-4 to H.264 transcoding with frame rate reduction

In this paper, a temporal resolution reduction transcoding method that transforms an MPEG-4 video bitstream into an H.264 video bitstream is proposed. The block modes and motion vectors in the MPEG-4 bitstream are utilized in the H.264 encoder for the block mode conversion and motion vector interpolation methods. Four types of motion vector interpolation methods are proposed in order to avoid the use of brute-force motion estimation in H.264. According to the experimental results, the proposed methods achieve a 3∼4 times improvement in the computational complexity compared to the cascade pixel-domain transcoding method, while the PSNR (peak signal to noise ratio) is degraded by 0.2∼0.9 dB depending on the bitrates.

Fast video transcoding from H.263 to H.264/MPEG-4 AVC

In the past 10 years detailed works on different video transcoders have been published. However, the new ITU-T Recommendation H.264—also adapted as ISO/IEC MPEG-4 Part 10 (AVC)—provides many new encoding options for the prediction processes that lead to difficulties for low complexity transcoding. In this work we present very fast transcoding techniques to convert H.263 bitstreams into H.264/AVC bitstreams. We will give reasoning, why the proposed pixel domain approach is advantageous in this scenario instead of using a DCT domain transcoder. Our approach results in less than 9% higher data rate at equivalent PSNR quality compared to a full-search approach. But this rate loss allows the reduction of the search complexity by a factor of over 200 for inter frames and still a reduction of over 70% for intra frames. A comparison to a fast search algorithm is given. We also provide simulation results that our algorithm works for transcoding MPEG-2 to H.264/AVC in the aimed scenario.

Low-complexity quantization for H.264/AVC

In this paper, an improved quantization technology with low-complexity is presented for H.264/AVC video codec. Multiplication factors of H.264/AVC quantizer are modified. Therefore, it is possible to reduce the bit width of the quantization and substitute large bit-width multiplier by some small bit-width adders without noticeable rate-distortion degradation in integrated circuits (ICs) design. Quantization error introduced by the modified multiplication factors is not only theoretically but also experimentally analyzed. Quantizer is optimized on register transfer level of IC design, and under the same cell CMOS technology, about 75.2% area and 76.3% dynamic power consumption are saved in each quantization unit on average compared with original H.264/AVC quantization. Experimental video coding results show that the Bjontegaard delta peak signal-to-noise ratio (PSNR) and Bjontegaard delta bit rate between the improved and original H.264/AVC quantization are very slight, which means that the improved quantization scheme is approximately the same as the original quantization scheme of H.264/AVC in rate-distortion performance.

Perfect requantization for video transcoding

Bit rate adaptation is one of the most important types of video transcoding. In this paper, we investigate certain critical points in the spectrum of rate shaping requests. We show that the selection of quantization step sizes may not have monotonic effects on rate-distortion characteristics in the transcoding sense. In other words, rate-distortion tradeoff for transcoding which operates on an already-compressed source can be different than that for regular encoding which operates on an original source. We show in a generic form that careful selections of the step size can lead to much improved performance for transcoding, especially when comparing to what would have been produced through a direct encoding. We demonstrate this unique rate-distortion characteristic through simulations as well as real transcoding scenario with H.264 sequences.

Mixed architectures for H.264/AVC digital video transrating

In this paper, we investigate transrating architectures for H.264/AVC video streams. Basic architectures are presented with their strengths and weaknesses. None of the existing architectures provide an appropriate solution for H.264/AVC transrating with an optimal balance between visual quality and complexity. In order to find such an appropriate solution, we propose the use of mixed transrating architectures. These architectures combine different transrating techniques which are applied depending on the picture/macroblock type. The intra-predicted pictures are decoded and re-encoded, while open-loop transrating or transrating with compensation is applied to motion-compensated pictures. Performance results show that the mixed architecture which applies spatial compensation to motion-compensated pictures gives rate-distortion results which approach the cascade of decoder and re-encoder with a complexity only slightly higher than the open-loop transrater. Adding temporal compensation for motion-compensated pictures further improves the visual quality, albeit to a lower extent, at the expense of increased complexity.

Distributed Video Coding using Turbo Trellis Coded Modulation

Distributed Video Coding (DVC) has been proposed for increasingly new application domains. This rise is apparently motivated by the very attractive features of its flexibility for building very low cost video encoders and the very high built-in error resilience when applied over noisy communication channels. Yet, the compression efficiency of DVC is notably lagging behind the state-of-the-art in video coding and compression, H.264/AVC in particular. In this context, a novel coding solution for DVC is presented in this paper, which promises to improve its rate-distortion (RD) performance towards the state-of-the-art. Here, Turbo Trellis Coded Modulation (TTCM), with its attractive coding gain in channel coding, is utilized and its resultant impact in both pixel domain and transform domain DVC framework is discussed herein. Simulations have shown a significant gain in the RD performance when compared with the state-of-the-art Turbo coding based DVC implementations.

An improved block size selection method based on macroblock movement characteristic

The H.264 video compression standard supports seven variable block sizes ranging from 4 × 4 to 16 × 16 for one Macro Block (MB) with 16 × 16 size to conduct motion estimation (ME) and compensation. This new feature achieves significant coding gain at the cost of huge computation complexity. Dozens of fast mode decision algorithms with fast block size selection have been proposed to reduce complexity. In this paper, we propose an improved fast block size selection method based on MB movement characteristic. The Motion Vector (MV) and block residual are employed to analyze the movement characteristic of one MB novelly. Then the movement characteristic is used to decide whether and how to merge or split the MB for encoding. Experimental results show that this method speeds up mode decision procedure dramatically with negligible compression performance degradation.

The training of Karhunen–Loève transform matrix and its application for H.264 intra coding

In H.264/AVC, 4 × 4 discrete cosine transform (DCT) is performed on the residual signals after intra prediction for decorrelation. Actually, residual blocks with different prediction modes exhibit different frequency characteristics. Therefore, the fixed transform matrix cannot match the energetic distribution of residual signals very well, which degrades the decorrelation performance. Fortunately, the energetic distributions of residual blocks with the same mode are relatively coincident, which makes it possible to train a universally good Karhunen–Loève transform (KLT) matrix for each mode. In this paper, an optimal frequency matching (OFM) algorithm is proposed to train KLT matrices for residual blocks and nine KLT matrices corresponding to nine prediction modes of 4 × 4 intra blocks are trained. Experimental results show that KLT with trained matrices yields a persistent gain over H.264 using 4 × 4 DCT with an average peak signal-to-noise ratio (PSNR) enhancement of 0.22dB and a maximum enhancement of 0.33dB.

Segmentation and recognition of motion capture data stream by classification

Three dimensional human motions recorded by motion capture and hand gestures recorded by using data gloves generate variable-length data streams. These data streams usually have dozens of attributes, and have different variations for similar motions. To segment and recognize motion streams, a classification-based approach is proposed in this paper. Classification feature vectors are extracted by utilizing singular value decompositions (SVD) of motion data. The extracted feature vectors capture the dominating geometric structures of motion data as revealed by SVD. Multi-class support vector machine (SVM) classifiers with class probability estimates are explored for classifying the feature vectors in order to segment and recognize motion streams. Experiments show that the proposed approach can find patterns in motion data streams with high accuracy.

Efficient video encryption scheme based on advanced video coding

A video encryption scheme combining with advanced video coding (AVC) is presented and analyzed in this paper, which is different from the ones used in MPEG1/2 video encryption. In the proposed scheme, the intra-prediction mode and motion vector difference are encrypted with the length-kept encryption algorithm (LKE) in order to keep the format compliance, and the residue data of the macroblocks are encrypted with the residue data encryption algorithm (RDE) in order to keep low cost. Additionally, a key distribution scheme is proposed to keep the robustness to transmission errors, which assigns sub-keys to different frames or slices independently. The encryption scheme’s security, time efficiency and error robustness are analyzed in detail. Experimental results show that the encryption scheme keeps file format unchanged, is secure against replacement attacks, is efficient in computing, and is robust to some transmission errors. These properties make it a suitable choice for real-time applications, such as secure IPTV, secure videoconference or mobile/wireless multimedia, etc.

Video frame rate up conversion under inconsistent camera motion

In this paper, we address the problem of video frame rate up-conversion (FRC) in the compressed domain. FRC is often recognized as video temporal interpolation. This problem is very challenging when targeted for video sequences with inconsistent camera and object motion, such as sports videos. A novel compressed domain motion compensation scheme is presented and applied in this paper, aiming at up-sampling frame rates in sports videos. MPEG-2 encoded motion vectors (MVs) are utilized as inputs in the proposed algorithm. The decoded MVs undergo a cumulative spatiotemporal interpolation. An iterative rejection scheme based on the dense motion vector field (MVF) and the generalized affine motion model is exploited to detect global camera motion. Subsequently, the foreground object separation is performed by additionally examining the temporal consistency of the output of iterative rejections. This consistency check process helps coalesce the resulting foreground blocks and weed out the unqualified blocks. Finally, different compensation strategies for the camera and object motions are applied to interpolate the new frames. Illustrative examples are provided to demonstrate the efficacy of the proposed approach. Experimental results are compared with the popular block and non-block based frame interpolation approaches.

Architecture of an application-specific processor for real-time implementation of H.264/AVC sub-pixel interpolation

This paper presents an efficient VLSI architecture for fast implementation of sub-pixel interpolation of H.264/AVC. Several optimization techniques at different design levels, such as parallel processing, vector register, pipeline architecture, and in-place computation, are utilized to reduce the number of memory access and accelerate the interpolation computations. The proposed application-specific processor can meet the real-time constraint of the sub-pixel interpolation algorithm for the 16:9 video format (4,690 × 2,304) at 30 frames per second (fps) at 100 MHz clock rate.

Fast adaptive termination mode selection for H.264 scalable video coding

Although scalable video coding can achieve coding efficiencies comparable with single layer video coding, its computational complexity is higher due to its additional inter-layer prediction process. This paper presents a fast adaptive termination algorithm for mode selection to increase its computation speed while attempting to maintain its coding efficiency. The developed algorithm consists of the following three main steps which are applied not only to the enhancement layer but also to the base layer: a prediction step based on neighboring macroblocks, a first round check step, and a second round check step or refinement if failure occurs during the first round check. Comparison results with the existing algorithms are provided. The results obtained on various video sequences show that the introduced algorithm achieves about one-third reduction in the computation speed while generating more or less the same video quality.

An improved variable-size block-matching algorithm

In this paper, we proposed an improved “bottom–up” variable-size block matching method. Different from previous work, the proposed method does not need any threshold during the matching, and we just keep all the motion vectors leading to the minimum matching error. A Marco-block mode prediction method is put forward to speed up the motion estimation procedure without introducing any loss to the prediction precision. The improved variable-size block matching algorithm can achieve exactly the same prediction precision as full-search based fixed-size block matching algorithm. In order to reduce the effect of illumination change on mode selection, we proposed an illumination removal method, which acts as a post-processing step to prevent the macro-blocks from over-splitting. Experiments show its encouraging performance.

Multimodal support to group dynamics

The complexity of group dynamics occurring in small group interactions often hinders the performance of teams. The availability of rich multimodal information about what is going on during the meeting makes it possible to explore the possibility of providing support to dysfunctional teams from facilitation to training sessions addressing both the individuals and the group as a whole. A necessary step in this direction is that of capturing and understanding group dynamics. In this paper, we discuss a particular scenario, in which meeting participants receive multimedia feedback on their relational behaviour, as a first step towards increasing self-awareness. We describe the background and the motivation for a coding scheme for annotating meeting recordings partially inspired by the Bales’ Interaction Process Analysis. This coding scheme was aimed at identifying suitable observable behavioural sequences. The study is complemented with an experimental investigation on the acceptability of such a service.

---