Adaptive video coding control for real-time H.264/AVC encoder

In this paper, we present a new adaptive video coding control for real-time H.264/AVC encoding system. The main techniques include: (1) the initial quantization parameter (QP) decision scheme is based on Laplacian of Gaussian (LoG) operators; (2) the MB-level QP calculation is based on the spatio-temporal correlation, in which the computation is less than the quadratic model used by H.264/AVC; (3) the adaptive GOP structure is proposed, in which the I-frame is adaptively replaced by an enhancement P-frame to improve the coding efficiency; (4) the scene change is detected with the complexity of adjacent inter-frames and the appropriate QP is re-calculated for the scene-change frame. The proposed algorithm is not only to save the computational complexity but also to improve coding quality. Compared to the JM12.4 reference under various sequences testing, the proposed algorithm can decrease coding time by 64.5% and improve PSNR by 1.52 dB while keeping the same bit-rate.     

A high-throughput ASIC processor for 8×8 transform coding in H.264/AVC

In July 2004, a new amendment called Fidelity Range Extensions (FRExt) was added to the H.264/AVC as a standardization initiative motivated by the rapidly growing demands when coding higher-fidelity video material. One improvement present in the FRExt is the inclusion of a new 8×8 integer transform that only makes use of additions and shifters to avoid mismatches between encoders and decoders. This paper presents a processor with pipeline architecture for real-time implementation of the complete process for the 8×8 Transform Coding in H.264: forward 8×8 integer transform, quantization and scaling, re-scaling, inverse 8×8 integer transform and reconstruction of the image block. This architecture has been conceived with the aim of achieving a high operation frequency and high throughput without increasing the hardware complexity. In order to achieve an efficient implementation, hardware solutions have been developed for the different circuit modules. 8×8 forward and inverse transforms are calculated using the separability property with architecture more suitable for pipeline schemes made up of two 1D processors and a transpose register array. New expressions for forward quantization and scaling are presented allowing efficient hardware implementation by avoiding the sign conversion. The inverse quantization has also been optimized in terms of hardware complexity by minimizing the involved arithmetic operations. Furthermore, an exhaustive analysis in the dynamic range of the datapath is made to fix the optimum bus widths with the aim of reducing the size of the circuit while avoiding overflow. Finally, the critical paths of the various computing units have been carefully analyzed and balanced using a pipeline scheme in order to maximize the operation frequency without introducing an excessive latency. A prototype with the proposed architecture has been synthesized in a 130 nm HCMOS technology process, which achieves a maximum speed of 330 MHz with a throughput of 2640 Mpixels/s.     

NEW FAST ALGORITHM OF 2-D DISCRETE COSINE TRANSFORM

In this paper, a new algorithm for the fast computation of a 2-D discrete cosine transform (DCT) is presented. It is shown that the N×N DCT, where N = 2m, can be computed using only N 1-D DCT's and additions, instead of using 2N 1-D DCT's as in the conventional row-column approach. Hence the total number of multiplications for the proposed algorithm is only half of that required for the row-column approach, and is also less than that of most of other fast algorithms, while the number of additions is almost comparable to that of others.     

A novel watermarking scheme for H.264/AVC video authentication

As the H.264/AVC-based video products become more and more popular, issues of copyright protection and authentication that are appropriate for this standard will be very important. In this paper, a content-based authentication watermarking scheme for H.264/AVC video is proposed. Considering the new feature of H.264/AVC, the content-based authentication code for spatial tampering is firstly generated using the reliable features extracted from video frame blocks. The authentication code, which can detect malicious manipulations but allow recompression, is embedded into the DCT coefficients in diagonal positions using a novel modulation method. Spatial tampering can be located by comparing the extracted and the original feature-based watermarks. In addition, combining ECC and interleaving coding, the frame index of each video frame is used as watermark information and embedded in the residual coefficients. Temporal tampering can be detected by the mismatch between the extracted and the observed frame index. Experimental results show that the proposed scheme can discriminate the malicious tampering from the mild signal processing. The tampered location can also be approximately determined according to the glide window and the predefined threshold.     

Transparent encryption techniques for H.264/AVC and H.264/SVC compressed video

Transparent encryption of video content requires to provide a video preview that is left in plaintext, while the enhancement information is encrypted. In this paper we propose three algorithms that provide transparent encryption. The first two ones are based on the idea of generating controlled drift in such a way as to obtain the desired quality level, while the third algorithm employs scalable video coding. We provide experimental results on several video sequences, as well as a security analysis, showing that the proposed algorithms provide an effective framework to perform transparent encryption.     

Edge-preserving Intra mode for efficient depth map coding based on H.264/AVC

Depth-image-based-rendering (DIBR) algorithms for 3D video communication systems based on the “multi-view video plus depth” format are very sensitive to the accuracy of depth information. Specifically, edge regions in the depth data should be preserved in the coding/decoding process to ensure good view synthesis performance, which directly affects the overall system performance. This paper proposes a novel scheme for edge-aware Intra depth compression based on the H.264/AVC framework enabled on both Intra (I) and Inter (P) slices. The proposed scheme includes a new Intra mode specifically targeted to depth macroblocks with arbitrarily shaped edges, which are typically not predicted well by the standard Intra modes of H.264/AVC and result in high rate–distortion costs. The proposed algorithm segments edge macroblocks into two regions each approximated by a flat surface. A binary mask identifying the two regions is defined and encoded by means of context-coding with adaptive template selection. As a novel contribution, the proposed mode allows exploiting the correlation with causal neighboring edge macroblocks to improve the performance of context-coding of binary masks and allow significant bit rate savings. The proposed method has been exhaustively compared with different state-of-the-art algorithms for edge-aware depth coding and the results highlight significant improvements in most of the cases, both in terms of reconstructed depth quality, view synthesis performance, and overall texture plus depth rate–distortion performance.     

基于H.264编码算法的高速CCD视频压缩系统设计

为了满足航空大面阵CCD相机视频数据高速、实时传输和存储的要求,本文设计了一种基于H.264视频编码算法的压缩系统。整个压缩系统分为CCD前端、视频压缩、视频显示、视频压缩码流存储以及压缩分析单元,视频压缩单元采用高性能视频专用DSP处理器TMS320DM642,软件平台采用在CCS3.1上使用C语言实现H.264压缩算法。为了使压缩算法高效快速的运行,本文使用了DSP/BIOS资源来管理软硬件工作。 为了高速交互数据,采用了EDMA高速搬运数据策略,进而保证了数据实时传输的需要。实验结果表明,本文提出的压缩系统可以稳定正常的工作,具有良好压缩性能,在压缩比40:1~10:1范围内,平均信噪比高于35dB,满足了航空CCD相机应用的需求。     

Requantization transcoding for H.264/AVC video coding

In this paper, efficient solutions for requantization transcoding in H.264/AVC are presented. By requantizing residual coefficients in the bitstream, different error components can appear in the transcoded video stream. Firstly, a requantization error is present due to successive quantization in encoder and transcoder. In addition to the requantization error, the loss of information caused by coarser quantization will propagate due to dependencies in the bitstream. Because of the use of intra prediction and motion-compensated prediction in H.264/AVC, both spatial and temporal drift propagation arise in transcoded H.264/AVC video streams. The spatial drift in intra-predicted blocks results from mismatches in the surrounding prediction pixels as a consequence of requantization. In this paper, both spatial and temporal drift components are analyzed. As is shown, spatial drift has a determining impact on the visual quality of transcoded video streams in H.264/AVC. In particular, this type of drift results in serious distortion and disturbing artifacts in the transcoded video stream. In order to avoid the spatially propagating distortion, we introduce transcoding architectures based on spatial compensation techniques. By combining the individual temporal and spatial compensation approaches and applying different techniques based on the picture and/or macroblock type, overall architectures are obtained that provide a trade-off between computational complexity and rate-distortion performance. The complexity of the presented architectures is significantly reduced when compared to cascaded decoder–encoder solutions, which are typically used for H.264/AVC transcoding. The reduction in complexity is particularly large for the solution which uses spatial compensation only. When compared to traditional solutions without spatial compensation, both visual and objective quality results are highly improved.     

Adaptive Scanning for H.264/AVC Intra Coding

In this letter, an adaptive scanning that improves intra coding efficiency in the H.264/AVC standard is proposed. The proposed adaptive scanning utilizes the prediction directions (modes) that include the horizontal and vertical edge information in a block. Depending on the prediction directions, the proposed method uses three scanning methods: zigzag scanning, horizontal scanning, and vertical scanning. In the proposed method, horizontal and vertical scanning are used in vertical and horizontal prediction modes, respectively, and the normal zigzag scanning in the H.264 standard is used in all other intra prediction modes. The proposed method reduces the bit rate by approximately 2.5% compared with H.264/AVC, without the degradation of video quality.     

SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming

The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of the video streaming application over the error-prone network, because it does not consider the transmission induced distortion in the encoding process. In this paper, a SSIM-based error-resilient RDO scheme for H.264/AVC is proposed to improve the wireless video streaming performance. Firstly, with the help of the SSE-based RDO, we present a low-complexity Lagrange multiplier decision method for the SSIM-based RDO video coding in the error-free environment. Then, the SSIM-based decoding distortion of the user end is estimated at the encoder and is correspondingly introduced into the RDO to involve the transmission induced distortion into the encoding process. Further, the Lagrange multiplier is theoretically derived to optimize the encoding mode selection in the error-resilient RDO process. Experimental results show that the proposed SSIM-based error-resilient RDO can obtain superior perceptual video quality (more structural information) to the traditional SSE-based error-resilient RDO for wireless video streaming at the same bit rate condition.     

An efficient 3-dimensional discrete wavelet transform architecture for video processing application

This paper presents an optimized 3-D Discrete Wavelet Transform (3-DDWT) architecture. 1-DDWT employed for the design of 3-DDWT architecture uses reduced lifting scheme approach. Further the architecture is optimized by applying block enabling technique, scaling, and rounding of the filter coefficients. The proposed architecture uses biorthogonal (9/7) wavelet filter. The architecture is modeled using Verilog HDL, simulated using ModelSim, synthesized using Xilinx ISE and finally implemented on Virtex-5 FPGA. The proposed 3-DDWT architecture has slice register utilization of 5%, operating frequency of 396 MHz and a power consumption of 0.45 W.     

Fast mode decision algorithm for intra prediction in H.264/AVC with integer transform and adaptive threshold

Compared with other existing video coding standards, H.264/AVC can achieve a significant improvement in compression performances. A robust criterion named the rate distortion optimization (RDO) is employed to select the optimal coding modes and motion vectors for each macroblock (MB), which achieves a high compression ratio while leading to a great increase in the complexity and computational load unfortunately. In this paper, a fast mode decision algorithm for H.264/AVC intra prediction based on integer transform and adaptive threshold is proposed. Before the intra prediction, integer transform operations on the original image are executed to find the directions of local textures. According to this direction, only a small part of the possible intra prediction modes are tested for RDO calculation at the first step. If the minimum mean absolute error (MMAE) of the reconstructed block corresponding to the best mode is smaller than an adaptive threshold which depends on the quantization parameter (QP), the RDO calculation is terminated. Otherwise, more possible modes need to be tested. The adaptive threshold aims to balance the compression performance and the computational load. Simulation results with various video sequences show that the fast mode decision algorithm proposed in this paper can accelerate the encoding speed significantly only with negligible PSNR loss or bit rate increment. This work is supported in part by China National Natural Science Foundation (CNSF) under Project No.60572045, the Ministry of Education of China Ph.D. Program Foundation under Project No.20050698033, and by a Cooperation Project (2005.7– 2007.7) with Microsoft Research Asia.     

New rate-distortion modeling and efficient rate control for H.264/AVC video coding

Rate control (RC) is crucial in controlling compression bit rates and qualities for networked video applications. In this paper, we propose a new rate-distortion (R-D) model and an efficient rate control scheme for H.264/AVC video coding, which elegantly resolve the inter-dependency problem between rate-distortion optimization and rate control by eliminating the need of coding complexity prediction for an inter-frame. The objective is to achieve accurate bit rate, obtain optimal video quality while reducing quality variations and simultaneously handling buffer fullness effectively. The proposed algorithm encapsulates a number of new features, including a coding complexity measure for intra-frames, a rate-distortion model, an accurate quantization parameter (QP) estimation for intra-frames, an incremental quantization parameter calculation method for inter-frames, a proportional+integral+derivative (PID) buffer controller, and an intelligent bit-allocation-balancing technique. Our experimental results demonstrate that the proposed scheme outperforms the JVT-G012 solution by providing accurate rate regulation, effectively reducing frame skipping, and finally improving coding quality by up to 1.80 dB.     

Restricted H.264/AVC video coding for privacy protected video scrambling

The issue of personal privacy has garnered significant attention with the extensive application of video surveillance systems. Privacy region scrambling is an effective method to protect privacy in video. To ensure that nonprivacy regions are not affected by scrambling, particular methods must be taken to prevent drift error in privacy protected video scrambling. However, existing methods have significantly reduced the coding efficiency. In this paper, we focus on improving coding efficiency while preventing drift error in privacy protected H.264/AVC video scrambling, which is the state-of-the-art coding standard. A restricted video coding scheme is proposed, which involves three parts of Mode Restricted Intra Prediction (MRIP), Search Window Restricted Motion Estimation (SWRME) and Boundary Strength Restricted Deblocking Filtering (BSRDF). Experimental results show that the proposed restricted video coding scheme prevents drift error with higher coding efficiency than others.     

Content-adaptive bitstream-layer model for coding distortion assessment of H.264/AVC networked video

Bitstream-layer models are designed to use the information extracted from both packet headers and payload for real-time and non-intrusive quality monitoring of networked video. This paper proposes a content-adaptive bitstream-layer (CABL) model for coding distortion assessment of H.264/AVC networked video. Firstly, the fundamental relationship between perceived coding distortion and quantization parameter (QP) is established. Then, considering the fact that the perceived coding distortion of a networked video significantly relies on both the spatial and temporal characteristics of video content, spatial and temporal complexities are incorporated in the proposed model. Assuming that the residuals before Discrete Cosine Transform (DCT) keep to the Laplace distribution, the scale parameters of the Laplace distribution are estimated utilizing QP and quantized coefficients on the basis of the Parseval theorem firstly. Then the spatial complexity is evaluated using QP and the scale parameters. Meanwhile, the temporal complexity is obtained using the weighted motion vectors (MV) considering the variations in temporal masking extent for high motion regions and low motion regions, respectively. Both the two characteristics of video content are extracted from the compressed bitstream without resorting to a complete decoding. Using content related information, the proposed model is able to adapt to different video contents. Experimental results show that the overall performance of CABL model significantly outperforms that of the P.1202.1 model and other coding distortion assessment models in terms of widely used performance criteria, including the Pearson Correlation Coefficient (PCC), the Spearman Rank Order Correlation Coefficient (SROCC), the Root-Mean-Squared Error (RMSE) and the Outlier Ratio (OR).     

Efficient entropy coding scheme for H.264/AVC lossless video coding

Context-based adaptive variable length coding (CAVLC) and context-based adaptive binary arithmetic coding (CABAC) are entropy coding methods employed in the H.264/AVC standard. Since these entropy coders are originally designed for encoding residual data, which are zigzag scanned and quantized transform coefficients, they cannot provide adequate coding performance for lossless video coding where residual data are not quantized transform coefficients, but the differential pixel values between the original and predicted pixel values. Therefore, considering the statistical characteristics of residual data in lossless video coding, we newly design each entropy coding method based on the conventional entropy coders in H.264/AVC. From the experimental result, we have verified that the proposed method provides not only positive bit-saving of 8% but also reduced computational complexity compared to the current H.264/AVC lossless coding mode.     

H.264的变换编码和量化过程分析

对H．264协议的变换编码和量化进行了理论分析，给出了具体实现过程，论证了这种变换和量化方式的特点、有效性及应用前景。     

基于矩的快速离散余弦变换处理机的设计及实现

阐述了基于矩的离散余弦变换算法和易于ＶＬＳＩ实现的脉动阵列算法结构,然后从软硬件结构划分、电路实现技术等方面探讨离散余弦变换处理机系统的设计思路。最后给出用矩实现的计算框图、电路实现框图以及外围驱动软件的结构设计。     

Mode dependent loop filter for intra prediction coding in H.264/AVC

In this paper, a high performance and low complexity loop filter is proposed for intra prediction coding. Although the deblocking loop filter (DLF) has achieved outstanding performance on suppressing quantization noise, it also induces details information loss because of the smoothing operation. To achieve better restoration performance, we propose a filter set named mode dependent loop filter (MDLF) which adaptively select the filter coefficients according to various local characteristics. In the homogeneous areas, the task of the filter emphasizes on smoothing the noise. In the heterogeneous areas, the proposed filter concentrates on preserving the details. Based on the spatial correlation assumption and statistical analysis, the intra mode combination is used to classify the training samples with different local characteristics. Then the classical least mean square error framework is employed to solve the coefficients for the proposed filter set. In this way, a more efficient adaptive loop filter scheme can be achieved for specific intra mode combination. Experiment results show that the proposed loop filter achieves superior coding gains compared to the H.264/AVC High Profile. Furthermore, relative to QALF+DLF, a comparable performance also can be achieved by the proposed MDLF with far less complexity increase.