Frame distortion estimation for unequal error protection methods in scalable video coding (SVC)

In many multimedia applications, coded video is transmitted over error prone heterogeneous networks. Because of the predictive mechanism used in video coding, transmission error would propagate temporally and spatially and would result in significant quality losses. In order to address this problem, different error resilience methods have been proposed. One of the techniques, which is commonly used in video streaming, is unequal error protection (UEP) of scalable video coding (SVC). In this technique, different independent layers of an SVC stream are protected differently and based on their importance by using forward error correction (FEC) codes. Accurately analyzing the importance or utility of each video part is a critical component and would lead to a better protection and higher quality of the received video. Calculation of the utility is usually based on multiple decoding of sub-bitstreams and is highly computationally complex. In this work, we propose an accurate low complexity utility estimation technique that can be used in different applications. This technique estimates the utility of each network abstraction layer (NAL) by considering the error propagation to future frames. We utilize this method in an UEP framework with the scalable extension of H.264/AVC codec and it achieves almost the same performance as highly complex estimation techniques (an average loss of 0.05 dB). Furthermore, we propose a low delay version of this technique that can be used in delay constrained application. The estimation accuracy and performance of our proposed technique are studied extensively.     

Distributed video coding based on lossy syndromes generated in hybrid pixel/transform domain

A recently proposed class of distributed source coding based video coders enables low-complexity compression and robust transmission over unreliable channels. These architectures process the video signal either in the pixel or in the transform domain generating some side information that permits a correct decoding of the coded image from a set of possible correlated sources. The approach proposed in this paper processes the video sequence both in the pixel and in the transform domain exploiting the advantages of both schemes and generating a set of lossy syndromes. The resulting video coding scheme requires a lower computational complexity at the decoder with respect to their transform-domain counterparts (like DISCOVER or PRISM) and provides a high compression gain and an increased robustness against channel losses.     

Mode dependent down-sampling and interpolation scheme for high efficiency video coding

In this paper, a mode dependent down-sampling and interpolation scheme is proposed to improve the coding efficiency of the intra prediction module. In the proposed method, we elaborately design the down-sampling structures and interpolation schemes for each directional intra prediction mode by minimizing the spatial prediction distance. The sampled pixels are predicted with a traditional directional intra prediction scheme, and the non-sampled pixels are predicted from the interpolation of their neighboring reconstructed sampling pixels. Both the residuals of the sampled and non-sampled pixels are encoded at last. Experimental results show that the proposed method achieves an average 7.52% bitrate reduction relative to KTA reference software. Since the down-sampling structure and interpolation method is only related to the intra mode, there is no additional overhead at the encoder.     

SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming

The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of the video streaming application over the error-prone network, because it does not consider the transmission induced distortion in the encoding process. In this paper, a SSIM-based error-resilient RDO scheme for H.264/AVC is proposed to improve the wireless video streaming performance. Firstly, with the help of the SSE-based RDO, we present a low-complexity Lagrange multiplier decision method for the SSIM-based RDO video coding in the error-free environment. Then, the SSIM-based decoding distortion of the user end is estimated at the encoder and is correspondingly introduced into the RDO to involve the transmission induced distortion into the encoding process. Further, the Lagrange multiplier is theoretically derived to optimize the encoding mode selection in the error-resilient RDO process. Experimental results show that the proposed SSIM-based error-resilient RDO can obtain superior perceptual video quality (more structural information) to the traditional SSE-based error-resilient RDO for wireless video streaming at the same bit rate condition.     

Perceptual-based distributed video coding

In this paper, we propose a perceptual-based distributed video coding (DVC) technique. Unlike traditional video codecs, DVC applies video prediction process at the decoder side using previously received frames. The predicted video frames (i.e., side information) contain prediction errors. The encoder then transmits error-correcting parity bits to the decoder to reconstruct the video frames from side information. However, channel codes based on i.i.d. noise models are not always efficient in correcting video prediction errors. In addition, some of the prediction errors do not cause perceptible visual distortions. From perceptual coding point of view, there is no need to correct such errors. This paper proposes a scheme for the decoder to perform perceptual quality analysis on the predicted side information. The decoder only requests parity bits to correct visually sensitive errors. More importantly, with the proposed technique, key frames can be encoded at higher rates while still maintaining consistent visual quality across the video sequence. As a result, even the objective PSNR measure of the decoded video sequence will increase too. Experimental results show that the proposed technique improves the R-D performance of a transform domain DVC codec both subjectively and objectively. Comparisons with a well-known DVC codec show that the proposed perceptual-based DVC coding scheme is very promising for distributed video coding framework.     

Two-dimensional reversible data hiding-based approach for intra-frame error concealment in H.264/AVC

Highly compressed video bit-stream is extremely sensitive to transmission error. A novel two-dimensional reversible data hiding-based approach for intra-frame error concealment is proposed, which aims at improving video quality at decoder when video bit-stream incur transmission errors. The scheme involves embedding the motion vector (MV) of a macroblock (MB) into other MB within the same intra-frame, and extracting the embedded MV from the received video frame for reconstruction of the corrupted MB. Based on the distribution of the motion vector data and the characteristic of histogram shifting, a specific two-dimensional reversible data hiding mechanism is designed. Consequently, the distortion of the marked video can be controlled at a low level. Experimental results show that the damaged macroblocks can be recovered with a higher quality using the reversible data hiding methodology, as compared to the non-reversible data hiding method. Furthermore, experimental results demonstrate that the proposed algorithm outperforms some state-of-the-art reversible data hiding error concealment schemes in improving the perceptual quality.     

Lifetime maximized data gathering in wireless sensor networks using limited-order distributed source coding

This paper addresses the problem of efficient data gathering in wireless sensor networks with a complexity constrained data gathering node. Due to the complexity constraint, the data gathering node employs an asymmetric DSC that (de)compresses the data of a given node exploiting its dependency with a limited number of other nodes. This is characterized in a DSC rate allocation structure that is referred to as limited-order DSC. Within this structure, we investigate the problem of rate allocation for the nodes to maximize the network lifetime. To this end, an algorithm is proposed that is proven optimal with polynomial complexity in terms of number of network nodes. Numerical results demonstrate that the algorithm, even with limited complexity, allows for exploiting most of the achievable compression gain.     

Motion-compensated DCT temporal filters for efficient spatio-temporal scalable video coding

Although most of the proposals for implementing motion-compensated temporal filtering (MCTF) schemes are based on the wavelet transform, in this paper, we propose an MCTF framework based on the discrete cosine transform (DCT). Using DCT decimation and interpolation, several temporal decomposition structures named motion-compensated DCT temporal filters (MCDCT-TF) are introduced. These structures are able to employ filters of any length with particular emphasis on 5/3 DCT and 7/4 DCT. The proposed MCDCT-TF and the two-dimensional (2D) DCT decimation technique are incorporated into H.264/AVC to provide spatio-temporal scalability. Compared with the current MCTF-based lifting schemes such as Haar, and 5/3 wavelet filters, simulation results show that the proposed MCDCT-TF utilizing longer tap DCT filters achieves a significant improvement in coding gain. The impact of odd/even group of frames, the decimation/interpolation ratios, and motion-compensated connectivity on the MCDCT-TF performance are also analyzed. Moreover, simulation results show that the performance of the presented scalable video coding is close to the single layer H.264/AVC and is slightly inferior to the temporal scalability supported in JSVM, the state-of-the-art scalable video coding standard, that gets its gain from Hierarchical B-pictures. However, our spatio-temporal coding scheme outperforms the spatio-temporal supported in JSVM even if it uses hierarchical B-pictures to improve its gain.     

A new distortion measure for motion estimation in motion-compensated hybrid video coding

In the video coding standards MPEG-x and H.26x, a motion-compensated prediction technique is used for enhancing the coding performance of bitrate reduction or peak signal to noise ratio (PSNR) improvement. This technique takes advantage of the correlation between consecutive frames in the time domain, which is relatively higher than that between adjacent blocks in the spatial domain. In order to utilize the correlation between consecutive frames, the conventional video coding standards have used the motion estimation (ME) and compensation technique, where the Sum of the Absolute Differences (SAD) is usually used as the distortion measure. The ME estimates the reference block that could minimize the residual signal between the current and reference blocks. However, the SAD is not appropriate to the specific sequences that have global or local illumination changes. In addition, the high-resolution video sequences have higher spatial correlation than the low-resolution video sequences in general. Therefore, a new distortion measure that can consider spatial and temporal correlation simultaneously may be helpful to enhance the coding performance. The proposed distortion measure searches for a reference block that minimizes the motion-compensated residual signal when the DC-component is predicted. In our proposed algorithm, the maximum BD-rate improvement is up to 13.6% for illumination-changed video sequences, and the average BD-rate improvement is 6.6% for various high-resolution video sequences in the baseline profile.     

x264视频编码器中参数设置对编码效率影响的研究

x264是基于H.264/MPEG-4 AVC视频压缩标准的视频压缩编码器,它提供了很多可以设置的参数来控制其编码效率。不同的参数设置影响着编码时间,压缩后的视频大小和视频质量。文章分别比较和分析了一些编码参数,包括图像组、帧内预测的宏块分割、帧间预测的宏块分割、参考帧数目和像素精度对视频的平均每帧编码时间、平均每帧大小和平均峰值信噪比带来的影响。     

Efficient rate control scheme for low bit rate H.264/AVC video coding

This article presents an efficient rate control scheme for H.264/AVC video coding in low bit rate environment. In the proposed scheme, an improved rate-distortion (RD) model by both analytical and empirical approaches is developed. It involves an enhanced mean absolute difference estimating method and a more rate-robust distortion model. Based on this RD model, an efficient macroblock-layer rate control scheme for H.264/AVC video coding is proposed. Experimental results show that this model encodes video sequences with higher peak signal-to-noise ratio gains and generates bit stream closer to the target rate.     

Turbo decoding for distributed source coding with combined distortion

Turbo codes are a promising technique for distributed source coding (DSC) in sensor networks because of their simple encoding implementation and high decoding performance. However, different from a conventional channel decoding scenario, decoding in DSC generally involves different types of distortion from physical and virtual channels. In this letter, we adapt the turbo decoding to handle data mixed with two types of distortion simultaneously. By adjusting the channel reliability values and the calculation of extrinsic information, it will be shown to what extent the decoding with the adaptation outperforms one considering single type of distortion only. Moreover, the adaptation, matching well with the realistic DSC scenario, only requires a slight change for an existing turbo decoding hardware unit.     

Dynamic macroblock wavefront parallelism for parallel video coding

Wavefront parallelism is effective for parallel video encoding thanks to its merits of low latency, no quality loss and high degree of parallelism. In traditional video encoders, macroblock row wavefront (MRW) parallelism was widely adopted. However the performance of MRW is limited by workload unbalance and computing resource unbalance among multiple work threads. This paper proposes a new dynamic macroblock wavefront (DMW) parallelism to alleviate the limitations of MRW. In DMW, the available macroblocks (MBs) are scheduled to work threads MB by MB other than MB row by MB row; and after one MB is encoded by a work thread, the MB on its right (if available) has the highest priority to be scheduled to avoid synchronization delay. Experimental results demonstrate that video encoder with traditional MRW can be accelerated by more than 10% with our proposed DMW. When hyper-threading is used, the advantage of DMW is more prominent.     

Multiview coding and error correction coding for 3D video over noisy channels

We consider the joint source–channel coding problem of stereo video transmitted over AWGN and flat Rayleigh fading channels. Multiview coding (MVC) is used to encode the source, as well as a type of spatial scalable MVC. Our goal is to minimize the total number of bits, which is the sum of the number of source bits and the number of forward error correction bits, under the constraints that the quality of the left and right views must each be greater than predetermined PSNR thresholds at the receiver. We first consider symmetric coding, for which the quality thresholds are equal. Following binocular suppression theory, we also consider asymmetric coding, for which the quality thresholds are unequal. The optimization problem is solved using both equal error protection (EEP) and a proposed unequal error protection (UEP) scheme. An estimate of the expected end-to-end distortion of the two views is formulated for a packetized MVC bitstream over a noisy channel. The UEP algorithm uses these estimates for packet rate allocation. Results for various scenarios, including non-scalable/scalable MVC, symmetric/asymmetric coding, and UEP/EEP, are provided for both AWGN and flat Rayleigh fading channels. The UEP bit savings compared to EEP are given, and the performances of different scenarios are compared for a set of stereo video sequences.     

分布式信源编码错误漂移消除机制的研究

传统视频编码标准由于具有较高的视频压缩效率而得到广泛使用,但是该方法存在错误漂移的现象,而分布式信源编码是当前解决这一现象行之有效的方法。目前提出的错误漂移消除机制采用固定间隔的方法,即每隔几帧选取一帧用于阻止错误漂移,但是该方法在选取视频帧的时候没有考虑被选取视频帧与前后帧的相关性,缺乏灵活性,针对这一问题提出了基于相关性的错误漂移消除选取机制。实验结果证明,该方法可以获得较好的解码效果。     

视频感兴趣区域快速提取与编码算法

如何在复杂度和时延受限的条件下,减少时域、空域和统计冗余,获得率失真性能的最优化是视频编码设计的核心问题。作为最新的视频编码标准,H.264在取得更高压缩率的同时,编码复杂度也随之剧增,限制了其在实时环境中的应用。本文在传统视频编码框架基础上,结合人类视觉系统感知特征,提出一种基于编码信息的视频感兴趣区域快速提取算法和优化编码方案。仿真实验结果表明,本文提出的编码策略在重建视频质量基本无损失和维持原有码率的提前下,将H.264编码时间平均节省52.55%,并能够与其它快速编码技术相结合,进一步提高H.264编码速度,以更好的适应异构网络及多样性终端环境,满足日益丰富的多媒体与通信业务需求。     

基于半脆弱水印的H.264/AVC视频流的内容级认证

针对低码率的H.264/AVC视频流,提出了一种基于半脆弱水印的内容级视频认证方案.利用I帧内块组之间能量不变的特点构建基于内容的特征码,并根据DCT系数特点,通过符号编码方式嵌入到幅值较小的DCT系数.为了保持嵌入前后视频流码率的基本恒定,引入了误差准则函数.此外,根据P帧中低频AC系数的正负符号统计特性,将帧号嵌入到AC系数.通过对提取出的水印信息和重构的认证码进行比较可实现篡改检测.仿真实验结果验证了本方案的有效性.     

Transparent encryption techniques for H.264/AVC and H.264/SVC compressed video

Transparent encryption of video content requires to provide a video preview that is left in plaintext, while the enhancement information is encrypted. In this paper we propose three algorithms that provide transparent encryption. The first two ones are based on the idea of generating controlled drift in such a way as to obtain the desired quality level, while the third algorithm employs scalable video coding. We provide experimental results on several video sequences, as well as a security analysis, showing that the proposed algorithms provide an effective framework to perform transparent encryption.     

Joint Source/Channel Coding Based on Two‐Dimensional Optimization for Scalable H.264/AVC Video

The scalable extension of the H.264/AVC video coding standard (SVC) demonstrates superb adaptability in video communications. Joint source and channel coding (JSCC) has been shown to be very effective for such scalable video consisting of parts of different significance. In this paper, a new JSCC scheme for SVC transmission over packet loss channels is proposed which performs two‐dimensional optimization on the quality layers of each frame in a rate‐distortion (R‐D) sense as well as on the temporal hierarchical structure of frames under dependency constraints. To compute the end‐to‐end R‐D points of a frame, a novel reduced trellis algorithm is developed with a significant reduction of complexity from the existing Viterbi‐based algorithm. The R‐D points of frames are sorted under the hierarchical dependency constraints and optimal JSCC solution is obtained in terms of the best R‐D performance. Experimental results show that our scheme outperforms the existing scheme of [13] with average quality gains of 0.26 dB and 0.22 dB for progressive and non‐progressive modes respectively.