实时3DV系统中面向虚拟视绘制的快速深度编码

在基于深度的三维视频系统中,合成视的质量与深度图的质量息息相关。为加快深度数据的压缩速度并保证重建虚拟视点图像的质量,提出了一种面向虚拟视点绘制的快速深度编码算法。根据深度数据的特点及深度失真对绘制视质量的影响,将深度图像划分为边缘区域与平坦区域,并相应地为深度编码中的宏块模式选择设计了两种搜索策略。对边缘区域及纹理复杂区域采用全搜索策略,对平坦区域及低纹理区域采用SKIP模式、帧间16×16模式和帧内模式搜索。实验结果表明,与X264编码方案相比,本文方法在保证虚拟绘制图像质量与编码码率不变的前提下,显著提高了编码速率,可应用于实时的基于深度的三维视频系统中。      

Adaptive depth truncation filter for MVC based compressed depth image

In multiview video plus depth (MVD) format, virtual views are generated from decoded texture videos with corresponding decoded depth images through depth image based rendering (DIBR). 3DV-ATM is a reference model for the H.264/AVC based multiview video coding (MVC) and aims at achieving high coding efficiency for 3D video in MVD format. Depth images are first downsampled then coded by 3DV-ATM. However, sharp object boundary characteristic of depth images does not well match with the transform coding based nature of H.264/AVC in 3DV-ATM. Depth boundaries are often blurred with ringing artifacts in the decoded depth images that result in noticeable artifacts in synthesized virtual views. This paper presents a low complexity adaptive depth truncation filter to recover the sharp object boundaries of the depth images using adaptive block repositioning and expansion for increasing the depth values refinement accuracy. This new approach is very efficient and can avoid false depth boundary refinement when block boundaries lie around the depth edge regions and ensure sufficient information within the processing block for depth layers classification. Experimental results demonstrate that the sharp depth edges can be recovered using the proposed filter and boundary artifacts in the synthesized views can be removed. The proposed method can provide improvement up to 3.25 dB in the depth map enhancement and bitrate reduction of 3.06% in the synthesized views.     

基于ZIG-ZAG交织的H.264/AVC容错编码算法

本文提出了一种基于ZIG-ZAG交织的灵活宏块排列算法ZFMO,按ZIG-ZAG扫描方式沿次对角线方向上交织排列宏块,从而在低位率下达到了编码效率和容错性能的最佳平衡点.实验结果表明,ZFMO在低丢包率的网络环境下比当前H.264/AVC的推荐算法具有更好的编码效率和容错性能.经过率失真优化后的ZFMO算法性能进一步提高.     

Efficient 45nm ASIC Architecture for Full-Search Free Intra Prediction in Real-Time H.264/AVC Decoder

The standard H.264/AVC Intra frame encoding process has several data dependent and computational intensive coding methodologies that limit the overall encoding speed. It causes not only a high degree of computational complexity but also an unacceptable delay especially for the real-time video applications. Based on DCT properties and spatial activity analysis, low power hardware architecture for high throughput Full-Search Free (FSF) Intra mode selection and direction prediction algorithm is proposed. The FSF Intra prediction Algorithm significantly reduces the computational complexity and the processing run-time required for the H.264/AVC Intra frame prediction process. The ASIC implementation for the proposed architecture is carried out and synthesizing results are obtained. The heavily tested 45nm ASIC design is able to achieve an operating frequency of 140 MHz while limiting the overall power consumption to 9.01 mW, which nominates our proposed FSF Intra prediction architecture for interactive real-time H.264/AVC mobile video decoders.     

Efficient error resilient algorithm for H.264/AVC: mobility management in wireless video streaming

The H.264/AVC standard introduces enhanced error robustness capabilities enabling resilient and reliable transmission of compressed video signals over wireless lossy packet networks. Those robustness capabilities are achieved by integrating some new error resilience tools that are essential for a proper delivery of real-time video services. Those tools include the Intra Refreshing (IR), Arbitrary Slice Ordering (ASO), Sequence Picture Parameter Sets (PPS), Redundant Slices (RS) tools and Flexible Macroblock Ordering (FMO). This paper presents an error resilient algorithm in wireless H.264/AVC streaming. The proposed method merges Reference Frame Selection (RFS), Intra Redundancy Slice and Adaptive Intra Refreshment techniques in order to prevent temporal error propagation in error-phone wireless video streaming. The coding standards only specify the decoding process and the bitstream syntax to allow considerable flexibility for the designers to optimize the encoder for coding performance improvement and complexity reduction. Performance evaluations demonstrate that the proposed encoding algorithm outperforms the conventional H.264/AVC standard. Both subjective and objective visual quality comparative study has been also carried out in order to validate the proposed approach. The proposed method can be used and integrated into H264/AVC without violating the standard.     

Multiview depth coding based on combined color/depth segmentation

In this paper, a new coding method for multiview depth video is presented. Considering the smooth structure and sharp edges of depth maps, a segmentation based approach is proposed. This allows further preserving the depth contours thus introducing fewer artifacts in the depth perception of the video. To reduce the cost associated with partition coding, an approximation of the depth partition is built using the decoded color view segmentation. This approximation is refined by sending some complementary information about the relevant differences between color and depth partitions. For coding the depth content of each region, a decomposition into orthogonal basis is used in this paper although similar decompositions may be also employed. Experimental results show that the proposed segmentation based depth coding method outperforms H.264/AVC and H.264/MVC by more than 2 dB at similar bitrates.     

General Architecture for MPEG-2/H.263/H.264/AVC to H.264/AVC Intra Frame Transcoding

The latest international video-coding standard H.264/AVC significantly achieves better coding performance compared to prior video coding standards such as MPEG-2 and H.263, which have been widely used in today’s digital video applications. To provide the interoperability between different coding standards, this paper proposes an efficient architecture for MPEG-2/H.263/H.264/AVC to H.264/AVC intra frame transcoding, using the original information such as discrete cosine transform (DCT) coefficients and coded mode type. Low-frequency components of DCT coefficients and a novel rate distortion cost function are used to select a set of candidate modes for rate distortion optimization (RDO) decision. For H.263 and H.264/AVC, a mode refinement scheme is utilized to eliminate unlikely modes before RDO mode decision, based on coded mode information. The experimental results, conducted on JM12.2 with fast C8MB mode decision, reveal that average 58%, 59% and 60% of computation (re-encoding) time can be saved for MPEG-2, H.263, H.264/AVC to H.264/AVC intra frame transcodings respectively, while preserving good coding performance when compared with complex cascaded pixel domain transcoding (CCPDT); or average 88% (a speed up factor of 8) when compared with CCPDT without considering fast C8MB. The proposed algorithm for H.264/AVC homogeneous transcoding is also compared to the simple cascaded pixel domain transcoding (with original mode reuse). The results of this comparison indicate that the proposed algorithm significantly outperforms the mode reuse algorithm in coding performance, with only slightly higher computation.     

Data-parallel intra decoding for block-based image and video coding on massively parallel architectures

With the increasing number of processor cores available in modern computing architectures, task or data parallelism is required to maximally exploit the available hardware and achieve optimal processing speed. Current state-of-the-art data-parallel processing methods for decoding image and video bitstreams are limited in parallelism by dependencies introduced by the coding tools and the number of synchronization points introduced by these dependencies, only allowing task or coarse-grain data parallelism. In particular, entropy decoding and data prediction are bottleneck coding tools for parallel image and video decoding. We propose a new data-parallel processing scheme for block-based intra sample and coefficient prediction that allows fine-grain parallelism and is suitable for integration in current and future state-of-the-art image and video codecs. Our prediction scheme enables maximum concurrency, independent of slice or tile configuration, while minimizing synchronization points. This paper describes our data-parallel processing scheme for one- and two-dimensional prediction and investigates its application to block-based image and video codecs using JPEG XR and H.264/AVC Intra as a starting point. We show how our scheme enables faster decoding than the state-of-the-art wavefront method with speedup factors of up to 21.5 and 7.9 for JPEG XR and H.264/AVC Intra coding tools respectively. Using the H.264/AVC Intra coding tool, we discuss the requirements of the algorithm and the impact on decoded image quality when these requirements are not met. Finally, we discuss the impact on coding rate in order to allow for optimal parallel intra decoding.     

Restricted H.264/AVC video coding for privacy protected video scrambling

The issue of personal privacy has garnered significant attention with the extensive application of video surveillance systems. Privacy region scrambling is an effective method to protect privacy in video. To ensure that nonprivacy regions are not affected by scrambling, particular methods must be taken to prevent drift error in privacy protected video scrambling. However, existing methods have significantly reduced the coding efficiency. In this paper, we focus on improving coding efficiency while preventing drift error in privacy protected H.264/AVC video scrambling, which is the state-of-the-art coding standard. A restricted video coding scheme is proposed, which involves three parts of Mode Restricted Intra Prediction (MRIP), Search Window Restricted Motion Estimation (SWRME) and Boundary Strength Restricted Deblocking Filtering (BSRDF). Experimental results show that the proposed restricted video coding scheme prevents drift error with higher coding efficiency than others.     

H.264/AVC标准中基于CABAC的数字视频加密研究

分析和总结了用于新一代视频编码标准H.264／AVC加密的候选域，在此基础上提出了一种新的基于CABAC（基于上下文的自适应二进制算术编码）的数字视频加密方案，并给出了2种安全加密操作：RCME（规则编码模式加密）和BCME（旁路编码模式加密）实现了残差系数码字、运动矢量差码字和帧内预测模式的加密保护。实验结果表明，该方案具有较好的安全性、编码效率和误码顽健性。     

Rate control for consistent visual quality of H.264/AVC encoding

In this paper, a novel rate control scheme with sliding window basic unit is proposed to achieve consistent or smooth visual quality for H.264/AVC based video streaming. A sliding window consists of a group of successive frames and moves forward by one frame each time. To make the sliding window scheme possible for real-time video streaming, the initial encoder delay inherently in a video streaming system is utilized to generate all the bits of a window in advance, so that these bits for transmission are ready before their due time. The use of initial encoder delay does not introduce any additional delay in video streaming but benefits visual quality as compared to traditional one-pass rate control algorithms of H.264/AVC. Then, a Sliding Window Buffer Checking (SWBC) algorithm is proposed for buffer control at sliding window level and it accords with traditional buffer measurement of H.264/AVC. Extensive experimental results exhibit that higher coding performance, consistent visual quality and compliant buffer constraint can be achieved by the proposed algorithm.     

Perceptually driven video error protection using a distributed source coding approach

In video communication systems, the video signals are typically compressed and sent to the decoder through an error-prone transmission channel that may corrupt the compressed signal, causing the degradation of the final decoded video quality. In this context, it is possible to enhance the error resilience of typical predictive video coding schemes using as inspiration principles and tools from an alternative video coding approach, the so-called Distributed Video Coding (DVC), based on the Distributed Source Coding (DSC) theory. Further improvements in the decoded video quality after error-prone transmission may also be obtained by considering the perceptual relevance of the video content, as distortions occurring in different regions of a picture have a different impact on the user's final experience. In this context, this paper proposes a Perceptually Driven Error Protection (PDEP) video coding solution that enhances the error resilience of a state-of-the-art H.264/AVC predictive video codec using DSC principles and perceptual considerations. To increase the H.264/AVC error resilience performance, the main technical novelties brought by the proposed video coding solution are: (i) design of an improved compressed domain perceptual classification mechanism; (ii) design of an improved transcoding tool for the DSC-based protection mechanism; and (iii) integration of a perceptual classification mechanism in an H.264/AVC compliant codec with a DSC-based error protection mechanism. The performance results obtained show that the proposed PDEP video codec provides a better performing alternative to traditional error protection video coding schemes, notably Forward Error Correction (FEC)-based schemes.     

An efficient macroblock-based diverse and flexible prediction modes selection for hyperspectral images coding

In this paper, an efficient macroblock-based diverse and flexible prediction modes selection algorithm is proposed for coding hyperspectral images, which is inspired by the prediction scheme of H264/AVC. Here, different modes are specified for the corresponding macroblocks (16×16 pixel regions of a band) of hyperspectral images other than the whole band image using only one reference band image for prediction. Only the 4×4 mode is employed for the intra-band prediction in view of the fact that correlation coefficients of pixels separated by not more than four pixels in the spatial domain are greater than 0.65 at most cases. The optimal reference band is determined by the fast reference band selection algorithm; thereafter, the best partition of the candidate macroblock in the optimal reference band is further selected for inter-band prediction of the current macroblock. Thus, the stronger correlation in the spectral direction or in the spatial domain is utilized for the prediction of the given macroblock. With a comparably low memory requirement, the prediction coding scheme is proposed to speed up the implemental process using the fast reference band selection algorithm, the integer DCT and the quantization, which just needs the multiplication and bit-shifts operations. Several AVIRIS images are used to evaluate the performance of the algorithm. The proposed scheme outperforms the state-of-the-art 3D-based compression algorithms at lower rates. Moreover, compared with the method by using all the prediction modes of H.264/AVC, about 80% encoding time can be saved by our method under the same experimental condition.     

Efficient entropy coding scheme for H.264/AVC lossless video coding

Context-based adaptive variable length coding (CAVLC) and context-based adaptive binary arithmetic coding (CABAC) are entropy coding methods employed in the H.264/AVC standard. Since these entropy coders are originally designed for encoding residual data, which are zigzag scanned and quantized transform coefficients, they cannot provide adequate coding performance for lossless video coding where residual data are not quantized transform coefficients, but the differential pixel values between the original and predicted pixel values. Therefore, considering the statistical characteristics of residual data in lossless video coding, we newly design each entropy coding method based on the conventional entropy coders in H.264/AVC. From the experimental result, we have verified that the proposed method provides not only positive bit-saving of 8% but also reduced computational complexity compared to the current H.264/AVC lossless coding mode.     

Novel H.264/AVC entropy coding mode decision

In this work, we propose a novel entropy coding mode decision algorithm to balance the tradeoff between the rate-distortion (R-D) performance and the entropy decoding complexity for the H.264/AVC video coding standard. Context-based adaptive binary arithmetic coding (CABAC), context-based adaptive variable length coding (CAVLC), and universal variable length coding (UVLC) are three entropy coding tools adopted by H.264/AVC. CABAC can be used to encode the texture and the header data while CAVLC and UVLC are employed to encode the texture and the header data, respectively. Although CABAC can provide better R-D performance than CAVLC/UVLC, its decoding complexity is higher. Thus, by taking the entropy decoding complexity into account, CABAC may not be the best tool, which motivates us to examine the entropy coding mode decision problem in depth. It will be shown experimentally that the proposed mode decision algorithm can help the encoder generate the bit streams that can be decoded at much lower complexity with little R-D performance loss.     

一种面向H.264/AVC的码率控制算法

码率控制是视频编码中非常重要的技术之一,任何标准离开码率控制其应用都会受到限制.H.264/AVC是目前最新的视频编码标准,本文根据H.264/AVC编码标准的特性及其HRD部分对码率控制的要求,提出了一种新的适合H.264/AVC的码率控制算法,该算法实现了率失真优化与码率控制的结合,使得在达到码率控制的同时也能保证较高的编码效率,同时在码率控制的过程中根据HRD缓冲区状态进行位分配调整,保证了编解码缓冲区既不上溢又不下溢.该算法作为技术提案已被H.264/AVC接受,并集成到H.264/AVC的校验模型软件中.     

Adaptive Pre‐/Post‐Filters for NRT‐Based Stereoscopic Video Coding

Non‐real‐time delivery of stereoscopic video has been considered as a service scenario for 3DTV to overcome the limited bandwidth in the terrestrial digital television system. A hybrid codec combining MPEG‐2 and H.264/AVC has been suggested for the compression of stereoscopic video for 3DTV. In this paper, we propose a stereoscopic video coding scheme using adaptive pre‐/post‐filters (APPF) to improve the quality of 3D video while retaining compatibility with legacy video coding standards. The APPF are applied adaptively to blocks of various sizes determined by the macroblock coding mode and reference frame index. Experiment results show that the proposed method achieves up to 24.86% bit rate savings relative to a hybrid codec of MPEG‐2 and H.264/AVC including the inter‐view prediction.     

Fast Interframe mode decision algorithm based on mode mapping and MB activity for MPEG-2 to H.264/AVC transcoding

Recently the latest video coding standard H.264/AVC is widely used for the mobile and low bitrate video codec in the various multimedia terminals. On the other hand, the MPEG-2 MP@HL codec has become the center of digital video contents since it is the standard codec for the Digital TV (DTV). To provide the bridge between the contents in MPEG-2 and mobile terminals, the transcoding of MPEG-2 contents into H.264/AVC format is an inevitable technology in the digital video market. The main bottleneck in the process lies in the computational complexity. In H.264/AVC, the variable block size (VBS) mode decision (MD) is used in the Interframe for the improved performance in the motion compensated prediction. For the macroblock (MB) which cannot be accurately predicted with one motion vector (MV), it is partitioned into smaller blocks and predicted with different MVs. In addition, SKIP and Intra modes are also permitted in the Interframe MD of H.264/AVC to further ameliorate the encoding performance. With the VBS MD technology, the Inter prediction accuracy can be improved significantly. However, the incidental side-effect is the high computational complexity. In this paper, we propose a fast Interframe MD algorithm for MPEG-2 to H.264/AVC transcoding. The relationships between SKIP and Intra modes are detected at first to map these two kinds of modes directly from MPEG-2 to H.264/AVC. And then the MB activity will be scaled by the residual DCT energy obtained from the MPEG-2 decoding process to estimate the block sizes of the MB mode for H.264/AVC Interframe MD. In our proposed method, the original redundant candidate modes can be eliminated effectively, resulting in the reduction of the computational complexity. It can reduce about 85% Rate-to-Distortion Cost (RDCost) computing and 45% entire processing time compared with the well-known cascaded transcoder while maintaining the video quality.     

一种基于缩减栅格算法的SVC联合信源/信道编码方法

 针对H.264/AVC可分级编码扩充标准(SVC),本文提出了一种联合信源/信道编码(JSCC)的视频传输方法.该方法采用一种新型的缩减栅格算法与拉格朗日优化技术对SVC数据与差错控制保护级别进行最优分配.本文提出的缩减栅格算法采用疏散式栅格结构,实行依层计算与缩减队列的策略,计算效率比同类高效算法提高了约20~50倍.大量仿真数据表明,采用本文高效算法的JSCC方法在各种信道条件下达到同类方法相同的优化性能.     

Post‐processing of 3D Video Extension of H.264/AVC for a Quality Enhancement of Synthesized View Sequences

Since July of 2012, the 3D video extension of H.264/AVC has been under development to support the multi‐view video plus depth format. In 3D video applications such as multi‐view and free‐view point applications, synthesized views are generated using coded texture video and coded depth video. Such synthesized views can be distorted by quantization noise and inaccuracy of 3D wrapping positions, thus it is important to improve their quality where possible. To achieve this, the relationship among the depth video, texture video, and synthesized view is investigated herein. Based on this investigation, an edge noise suppression filtering process to preserve the edges of the depth video and a method based on a total variation approach to maximum a posteriori probability estimates for reducing the quantization noise of the coded texture video. The experiment results show that the proposed methods improve the peak signal‐to‐noise ratio and visual quality of a synthesized view compared to a synthesized view without post processing methods.