Error Concealment Using Intra‐Mode Information Included in H.264/AVC‐Coded Bitstream

The H.264/AVC standard has adopted new coding tools such as intra‐prediction, variable block size, motion estimation with quarter‐pixel‐accuracy, loop filter, and so on. The adoption of these tools enables an H.264/AVC‐coded bitstream to have more information than was possible with previous standards. In this paper, we propose an effective spatial error concealment method with low complexity in H.264/AVC intra‐frame. From information included in an H.264/AVC‐coded bitstream, we use prediction modes of intra‐blocks to recover a damaged block. This is because the prediction direction in each prediction mode is highly correlated to the edge direction. We first estimate the edge direction of a damaged block using the prediction modes of the intra‐blocks adjacent to a damaged block and classify the area inside the damaged block into edge and flat areas. Our method then recovers pixel values in the edge area using edge‐directed interpolation, and recovers pixel values in the flat area using weighted interpolation. Simulation results show that the proposed method yields better video quality than conventional approaches.     

Perceptual-based distributed video coding

In this paper, we propose a perceptual-based distributed video coding (DVC) technique. Unlike traditional video codecs, DVC applies video prediction process at the decoder side using previously received frames. The predicted video frames (i.e., side information) contain prediction errors. The encoder then transmits error-correcting parity bits to the decoder to reconstruct the video frames from side information. However, channel codes based on i.i.d. noise models are not always efficient in correcting video prediction errors. In addition, some of the prediction errors do not cause perceptible visual distortions. From perceptual coding point of view, there is no need to correct such errors. This paper proposes a scheme for the decoder to perform perceptual quality analysis on the predicted side information. The decoder only requests parity bits to correct visually sensitive errors. More importantly, with the proposed technique, key frames can be encoded at higher rates while still maintaining consistent visual quality across the video sequence. As a result, even the objective PSNR measure of the decoded video sequence will increase too. Experimental results show that the proposed technique improves the R-D performance of a transform domain DVC codec both subjectively and objectively. Comparisons with a well-known DVC codec show that the proposed perceptual-based DVC coding scheme is very promising for distributed video coding framework.     

Scalable Interframe Wavelet Coding with Low Complex Spatial Wavelet Transform

In the decoding process associated with interframe wavelet coding, the inverse wavelet transform requires high computational complexity. However, as video technology starts to pervade all aspects of our lives, decoders are becoming required in various devices such as PDAs, notebooks, PCs, and set‐top boxes. Therefore, a decoder's complexity needs to be adapted to the processor's computational power, and consequently a low‐complexity codec is also required for scalable video coding. In this paper, we propose a method of controlling and lowering the complexity of the spatial wavelet transform while sustaining the same coding efficiency as that currently afforded. In addition, the proposed method may alleviate the ringing effect for slowly changing image sequences.     

基于AVS的帧场自适应编码算法研究

介绍了中国音视频编码技术标准AVS中的帧内预测和基于宏块的帧场自适应编码技术.通过分析其中的5种帧内预测模式的空间分布特点,提出了一种新的隔行视频帧内预测模式.实验表明该模式可以在标清、高清隔行视频的帧内预测上获得平均0.24 dB以上的信噪比增益,平均减少3.1%的码流,是一种有效提高编码效率的预测模式.     

最新一代视频压缩标准VP9综述

VP9是一个由谷歌开发的开放格式、无使用授权费的视频压缩标准,并做为谷歌推行WebM工程的一部分.对VP9标准的主要技术特征进行了分析,包括帧内预测、帧间预测、变换和量化、熵编码、环路滤波等,同时与其他先进的视频压缩标准HEVC进行了比较.     

Hardware‐Software Implementation of MPEG‐4 Video Codec

This paper presents an MPEG‐4 video codec, called MoVa, for video coding applications that adopts 3G‐324M. We designed MoVa to be optimal by embedding a cost‐effective ARM7TDMI core and partitioning it into hardwired blocks and firmware blocks to provide a reasonable tradeoff between computational requirements, power consumption, and programmability. Typical hardwired blocks are motion estimation and motion compensation, discrete cosine transform and quantization, and variable length coding and decoding, while intra refresh, rate control, error resilience, error concealment, etc. are implemented by software. MoVa has a pipeline structure and its operation is performed in four stages at encoding and in three stages at decoding. It meets the requirements of MPEG‐4 SP@L2 and can perform either 30 frames/s (fps) of QCIF or SQCIF, or 7.5 fps (in codec mode) to 15 fps (in encode/decode mode) of CIF at a maximum clock rate of 27 MHz for 128 kbps or 144 kbps. MoVa can be applied to many video systems requiring a high bit rate and various video formats, such as videophone, videoconferencing, surveillance, news, and entertainment.     

一种无帧内失真漂移的HEVC视频自适应隐写算法

随着数字视频的普及,HEVC(high efficiency video coding)视频隐写受到了越来越多的关注。残差系数域隐写具有高安全性和低码率增长的特点,然而,视频编码过程中重构的系数不满足块间相互独立的性质,导致修改视频残差系数往往会带来较为严重的失真漂移。本文提出了一种无帧内失真漂移的HEVC视频自适应隐写算法。首先,根据帧内预测方向和多系数修改的方法选择合适的载体,进而避免修改系数引起的失真漂移。然后,设计一种包含块内失真和修改载体后的系数分布的失真函数,指导校验网格编码(syndrome trellis codes, STC)对嵌入失真小的载体进行修改。最后,根据最小化失真代价将消息嵌入到满足特定条件的4×4亮度块的量化正弦变换(quantized discrete sinc transform, QDST)系数中。实验结果表明,所提出的视频隐写算法能够有效地避免帧内失真漂移,在获得良好的视觉感知质量的同时,能保证隐写算法的安全性。     

Perceptually driven video error protection using a distributed source coding approach

In video communication systems, the video signals are typically compressed and sent to the decoder through an error-prone transmission channel that may corrupt the compressed signal, causing the degradation of the final decoded video quality. In this context, it is possible to enhance the error resilience of typical predictive video coding schemes using as inspiration principles and tools from an alternative video coding approach, the so-called Distributed Video Coding (DVC), based on the Distributed Source Coding (DSC) theory. Further improvements in the decoded video quality after error-prone transmission may also be obtained by considering the perceptual relevance of the video content, as distortions occurring in different regions of a picture have a different impact on the user's final experience. In this context, this paper proposes a Perceptually Driven Error Protection (PDEP) video coding solution that enhances the error resilience of a state-of-the-art H.264/AVC predictive video codec using DSC principles and perceptual considerations. To increase the H.264/AVC error resilience performance, the main technical novelties brought by the proposed video coding solution are: (i) design of an improved compressed domain perceptual classification mechanism; (ii) design of an improved transcoding tool for the DSC-based protection mechanism; and (iii) integration of a perceptual classification mechanism in an H.264/AVC compliant codec with a DSC-based error protection mechanism. The performance results obtained show that the proposed PDEP video codec provides a better performing alternative to traditional error protection video coding schemes, notably Forward Error Correction (FEC)-based schemes.     

AVS2视频编码标准技术特色及应用

围绕视频编码核心技术,简单介绍AVS视频编码标准发展历程,详细介绍了最新一代AVS视频编码标准——AVS2(GB/T 33475.2-2016)的核心创新技术,主要包括:灵活的预测划分方式、多假设帧间预测、优化的层次变换设计和自适应环路滤波处理技术以及面向场景视频应用AVS2提出的基于场景建模的高效预测编码技术.和第一代AVS视频编码标准相比,AVS2编码效率提升一倍以上;和同期HEVC/H.265国际标准相比,AVS2在场景视频编码方面有显著优势.另外简单介绍了AVS标准在数字电视广播等行业中的应用情况.     

Temporal image sequence prediction using motion field interpolation

A new method for motion-compensated temporal prediction of image sequences is proposed. Motion vector fields in natural scenes should possess two basic properties. First, the field should be smoothly varying within moving objects to compensate for nonrigid or rotational motion, and scaling of objects. Second, the field should be discontinuous along the boundaries of the objects. In the proposed method the motion vector field is modelled using finite element methods and interpolated using adaptive interpolators to satisfy the above-stated requirements. This is particularly important when only very sparse estimates of motion vector fields are available in the decoder due to bit-rate constraints limiting the amount of overhead information that can be transmitted. The proposed prediction method can be applied for low-bit-rate video coding in conventional codecs based on motion-compensated prediction and transform coding, as well as in model-based codecs. The performance of the proposed method is compared with standard motion-compensated prediction based on block matching. It is shown that for simple video telephony scenes a reduction of more than 30% in the energy of the prediction error can be achieved with an unchanged number of transmitted motion vectors and with only a modest increase in computational complexity. When implemented in an H.261 codec the new prediction method can improve the peak SNR 1–2 dB producing a significant visual improvement.     

Multiview video compression with 1-D transforms

Many alternative transforms have been developed recently for improved compression of images, intra prediction residuals or motion-compensated prediction residuals. In this paper, we propose alternative transforms for multiview video coding. We analyze the spatial characteristics of disparity-compensated prediction residuals, and the analysis results show that many regions have 1-D signal characteristics, similar to previous findings for motion-compensated prediction residuals. Signals with such characteristics can be transformed more efficiently with transforms adapted to these characteristics and we propose to use 1-D transforms in the compression of disparity-compensated prediction residuals in multiview video coding. To show the compression gains achievable from using these transforms, we modify the reference software (JMVC) of the multiview video coding amendment to H.264/AVC so that each residual block can be transformed either with a 1-D transform or with the conventional 2-D Discrete Cosine Transform. Experimental results show that coding gains ranging from about 1–15% of Bjontegaard-Delta bitrate savings can be achieved.     

HEVC-based 3D holoscopic video coding using self-similarity compensated prediction

Holoscopic imaging, also known as integral, light field, and plenoptic imaging, is an appealing technology for glassless 3D video systems, which has recently emerged as a prospective candidate for future image and video applications, such as 3D television. However, to successfully introduce 3D holoscopic video applications into the market, adequate coding tools that can efficiently handle 3D holoscopic video are necessary. In this context, this paper discusses the requirements and challenges for 3D holoscopic video coding, and presents an efficient 3D holoscopic coding scheme based on High Efficiency Video Coding (HEVC). The proposed 3D holoscopic codec makes use of the self-similarity (SS) compensated prediction concept to efficiently explore the inherent correlation of the 3D holoscopic content in Intra- and Inter-coded frames, as well as a novel vector prediction scheme to take advantage of the peculiar characteristics of the SS prediction data. Extensive experiments were conducted, and have shown that the proposed solution is able to outperform HEVC as well as other coding solutions proposed in the literature. Moreover, a consistently better performance is also observed for a set of different quality metrics proposed in the literature for 3D holoscopic content, as well as for the visual quality of views synthesized from decompressed 3D holoscopic content.     

An automatic region-based video sequence codec based on fractal compression

In order to improve the performance of fractal video coding, we explore a novel fractal video sequences codec with automatic region-based functionality. To increase the quality of decoding image, intra frame coding, deblocking loop filter and sub-pixel block matching are applied to the codec. An efficient searching algorithm is used to increase the compression ratio and encoding speed. Automatic region-based fractal video sequences coding reduces coding stream greatly. Experimental results indicate that the proposed algorithm is more robust, and provides much less encoding time and bitrate while maintaining the quality of decompression image than the conventional CPM/NCIM method and other related references. We compare the proposed algorithm with three algorithms in Refs. [24], [25], [26], and the results of all these four algorithms are compared with H.264. The bitrate of the proposed algorithm is decreased by 0.11% and the other algorithms are increased by 4.29%, 6.85% and 11.62%, respectively. The average PSNR degradations of the four algorithms are 0.71 dB, 0.48 dB, 0.48 dB and 0.75 dB. So the bitrate of the proposed algorithm is decreased and the other algorithms are increased. At the meantime the compression time is reduced greatly, about 79.19% on average. The results indicate that, on average, the proposed automatic region-based fractal video sequences coding system can save compression time 48.97% and bitrate 52.02% with some image quality degradation in comparison with H.264, since they are all above 32 dB and the human eyes are insensitive to the differences.     

Fast mode decision algorithm for intra prediction in H.264/AVC with integer transform and adaptive threshold

Compared with other existing video coding standards, H.264/AVC can achieve a significant improvement in compression performances. A robust criterion named the rate distortion optimization (RDO) is employed to select the optimal coding modes and motion vectors for each macroblock (MB), which achieves a high compression ratio while leading to a great increase in the complexity and computational load unfortunately. In this paper, a fast mode decision algorithm for H.264/AVC intra prediction based on integer transform and adaptive threshold is proposed. Before the intra prediction, integer transform operations on the original image are executed to find the directions of local textures. According to this direction, only a small part of the possible intra prediction modes are tested for RDO calculation at the first step. If the minimum mean absolute error (MMAE) of the reconstructed block corresponding to the best mode is smaller than an adaptive threshold which depends on the quantization parameter (QP), the RDO calculation is terminated. Otherwise, more possible modes need to be tested. The adaptive threshold aims to balance the compression performance and the computational load. Simulation results with various video sequences show that the fast mode decision algorithm proposed in this paper can accelerate the encoding speed significantly only with negligible PSNR loss or bit rate increment. This work is supported in part by China National Natural Science Foundation (CNSF) under Project No.60572045, the Ministry of Education of China Ph.D. Program Foundation under Project No.20050698033, and by a Cooperation Project (2005.7– 2007.7) with Microsoft Research Asia.     

Scalable Extension of HEVC for Flexible High‐Quality Digital Video Content Services

This paper describes the scalable extension of High Efficiency Video Coding (HEVC) to provide flexible highquality digital video content services. The proposed scalable codec is designed on multi‐loop decoding architecture to support inter‐layer sample prediction and inter‐layer motion parameter prediction. Inter‐layer sample prediction is enabled by inserting the reconstructed picture of the reference layer (RL) into the decoded picture buffer of the enhancement layer (EL). To reduce the motion parameter redundancies between layers, the motion parameter of the RL is used as one of the candidates in merge mode and motion vector prediction in the EL. The proposed scalable extension can support scalabilities with minimum changes to the HEVC and provide average Bjøntegaard delta bitrate gains of about 24% for spatial scalability and of about 21% for SNR scalability compared to simulcast coding with HEVC.     

A pixel-wise directional intra prediction method

In a decade where the prominence of video applications has become increasingly prevalent, the optimization of video coding processes remains as important as ever. The directional intra prediction has been proved as an effective tool for intra picture coding. However, in a directional intra prediction based coding structure, there is a considerable gap between the transform efficiency of DCT and the optimal transform KLT. In this paper, we analyze these problems and propose a pixel-wise directional intra prediction (PDIP) method to solve this problem. This method exploits the reconstruction value of the adjacent pixels to predict the current pixel, and keeps the block-based lossy coding structure of H.264/AVC. The proposed method can significantly improve the coding efficiency with little decoding complexity increment.     

改进的H.264快速帧内预测选择算法

新一代的视频压缩标准H.264在视频数据压缩技术上取得了重大进展,但其算法复杂度明显提升,编码时间也较长。为降低H.264的计算复杂度和提高压缩效率,本文提出了一种基于变换域特性的变换域立体型（TDST）帧内预测快速选择算法,通过哈德码（Hadamard）变换的变换域特性对预测模式先进行选择,过滤掉大部分的预选模式,利...