Side information creation for efficient Wyner–Ziv video coding: Classifying and reviewing

Video coding technologies have played a major role in the explosion of large market digital video applications and services. In this context, the very popular MPEG-x and H-26x video coding standards adopted a predictive coding paradigm, where complex encoders exploit the data redundancy and irrelevancy to ‘control’ much simpler decoders. This codec paradigm fits well applications and services such as digital television and video storage where the decoder complexity is critical, but does not match well the requirements of emerging applications such as visual sensor networks where the encoder complexity is more critical. The Slepian–Wolf and Wyner–Ziv theorems brought the possibility to develop the so-called Wyner–Ziv video codecs, following a different coding paradigm where it is the task of the decoder, and not anymore of the encoder, to (fully or partly) exploit the video redundancy. Theoretically, Wyner–Ziv video coding does not incur in any compression performance penalty regarding the more traditional predictive coding paradigm (at least for certain conditions). In the context of Wyner–Ziv video codecs, the so-called side information, which is a decoder estimate of the original frame to code, plays a critical role in the overall compression performance. For this reason, much research effort has been invested in the past decade to develop increasingly more efficient side information creation methods. This paper has the main objective to review and evaluate the available side information methods after proposing a classification taxonomy to guide this review, allowing to achieve more solid conclusions and better identify the next relevant research challenges. After classifying the side information creation methods into four classes, notably guess, try, hint and learn, the review of the most important techniques in each class and the evaluation of some of them leads to the important conclusion that the side information creation methods provide better rate-distortion (RD) performance depending on the amount of temporal correlation in each video sequence. It became also clear that the best available Wyner–Ziv video coding solutions are almost systematically based on the learn approach. The best solutions are already able to systematically outperform the H.264/AVC Intra, and also the H.264/AVC zero-motion standard solutions for specific types of content.     

Encoder adaptable difference detection for low power video compression in surveillance system

As a state-of-the-art video compression technique, H.264/AVC has been deployed in many surveillance cameras to improve the compression efficiency. However, it induces very high coding complexity, and thus high power consumption. In this paper, a difference detection algorithm is proposed to reduce the computational complexity and power consumption in surveillance video compression by automatically distributing the video data to different modules of the video encoder according to their content similarity features. Without any requirement in changing the encoder hardware, the proposed algorithm provides high adaptability to be integrated into the existing H.264 video encoders. An average of over 82% of overall encoding complexity can be reduced regardless of whether or not the H.264 encoder itself has employed fast algorithms. No loss is observed in both subjective and objective video quality.     

Distributed Video Coding

Distributed coding is a new paradigm for video compression, based on Slepian and Wolf's and Wyner and Ziv's information-theoretic results from the 1970s. This paper reviews the recent development of practical distributed video coding schemes. Wyner-Ziv coding, i.e., lossy compression with receiver side information, enables low-complexity video encoding where the bulk of the computation is shifted to the decoder. Since the interframe dependence of the video sequence is exploited only at the decoder, an intraframe encoder can be combined with an interframe decoder. The rate-distortion performance is superior to conventional intraframe coding, but there is still a gap relative to conventional motion-compensated interframe coding. Wyner-Ziv coding is naturally robust against transmission errors and can be used for joint source-channel coding. A Wyner-Ziv MPEG encoder that protects the video waveform rather than the compressed bit stream achieves graceful degradation under deteriorating channel conditions without a layered signal representation.     

一种基于多视点视频的低复杂度自适应环路滤波算法

自适应环路滤波(ALF)是面向高清的多视点视频编码(MVC)中的一项新的滤波技术,虽然能提高视频压缩的主观质量和客观质量,但是其复杂度过高阻碍了其实时应用。本文提出了一种低复杂度的ALF算法,充分利用了视点间相关性和层间相关性以减少亮度和色度分量的分割次数,并且能自适应跳过大部分帧的块控制(DBC)处理过程。实验结果表明,所提出的算法能在保证视频主观质量和客观质量基本不变的情况下减少61%左右的ALF处理时间。因此,所提出的方法在编码效率和复杂度上做到了很好的折中处理,并且能很好地改善ALF的性能。     

Fast macroblock encoding algorithm based on rate-distortion activity for multiview video coding

Multiview video coding (MVC) is the appendix H of H.264/AVC, and it requires a great amount of time to compress multiple viewpoints׳ video with complex prediction structures. To reduce the whole computational complexity of MVC, this paper proposes a fast macroblock (MB) encoding algorithm based on rate-distortion (RD) activity, and it includes the fast mode decision and the fast motion/disparity estimation. First, the RD activity type of the current MB is calculated by utilizing the Skip/Direct RD cost and the average RD costs of classified MB modes. Then, through utilizing the RD activity type and RD costs of the estimated modes, the selection of candidate modes, the early decision of Skip/Direct mode, and the reduction of Inter8×8 mode estimation are all presented in the fast mode decision. By using the RD activity type and the correlations of vectors, the selection of search center and the prediction of search range are introduced in the fast motion/disparity estimation. In addition, the proposed algorithm can be applied to temporal and inter-view views as well as anchor and non-anchor frames. An experiment with a wide range of video scenes, camera setups and quantization parameters was implemented, and the results confirmed that the proposed algorithm can reduce the encoding time significantly while maintaining a similar RD performance as the original MVC encoder. Compared to the state-of-the-art algorithms, the proposed algorithm also demonstrated better performances in the various test cases.     

一种空间域Wyner-Ziv视频编码系统的性能改进算法

分布式视频编码是建立在Slepian-Wolf和Wyner-Ziv信息编码理论基础上的全新视频编码框架,具有编码复杂度低,编码效率较高,抗误码性能好的特点.本文首先简单介绍了一种典型的分布式视频编码实现方案——空间域Wyner-Ziv视频编码,随后提出一种空间域Wyner-Ziv视频编码系统的性能改进算法,该算法在不增加编码复杂度的基础上,在解码端利用双向运动估计预测获取更高质量的边信息,同时采用基于Huber-Markov随机场约束的联合迭代解码算法重建图像.实验结果表明,在相同的输出码流情况下,本文改进算法在解码端重建图像的峰值信噪比与空间域Wyner-Ziv视频编码算法相比平均提高2dB,并且主观效果有所改善.     

基于时空相关性的HEVC帧间模式决策快速算法

新一代的高效率视频编码标准HEVC采用编码树单元（CTU）四叉树划分技术和多达10种的帧间预测单元（PU）模式,有效地提高了编码压缩效率,但也极大地增加了编码计算复杂度。为了减少编码单元（CU）的划分次数和候选帧间PU模式个数,提出了一种基于时空相关性的帧间模式决策快速算法。首先,利用当前CTU与参考帧中相同位置CTU、当前帧中相邻CTU的深度信息时空相关性,有效预测当前CTU的深度范围。然后,通过分析当前CU与其父CU之间的最佳PU模式空间相关性,以及利用当前CU已估计PU模式的率失真代价,跳过当前CU的冗余帧间PU模式。实验结果表明,提出的算法与HEVC测试模型（HM）相比,在不同编码配置下降低了52%左右的编码时间,同时保持了良好的编码率失真性能;与打开快速算法选项的HM相比,所提算法进一步降低了30%左右的编码时间。     

基于DSP的AVS实时压缩系统设计与实现

介绍了一种基于TMS320DM642 DSP的视频压缩系统,该系统采用高性能的AVS视频编码标准,能实现视频图像的实时压缩、显示、存储和回放等功能。主要介绍了该系统的硬件结构原理,然后实现了系统软件的设计,并对AVS视频编码算法进行了优化。     

Multiview side information creation for efficient Wyner–Ziv video coding: Classifying and reviewing

In recent years, the interest in multiview video systems has increased. In these systems, a typical predictive coding approach exploits the inter-view correlation at a joint encoder, requiring the various cameras to communicate among them. However, many applications ask for simple sensing systems preventing the various cameras to communicate among them, and thus the adoption of a predictive coding approach. Wyner–Ziv (WZ) video coding is a promising solution for those applications since it is the WZ decoder task to (fully or partly) exploit the video redundancy. The rate-distortion (RD) performance of WZ video coding strongly depends on the quality of the so-called side information (SI), which is a decoder estimate of the original frame to code. In multiview WZ (MV-WZ) video coding, the target is to exploit in the best way the available correlation not only in time, as for the monoview case, but also between views. Thus, the multiview SI results from the fusion of a temporally created SI and an inter-view created SI. In this context, the main objective of this paper is to propose a classification taxonomy to organize the many inter-view SI creation and SI fusion techniques available in the literature and to review the most relevant techniques in each class. The inter-view SI creation techniques are classified into two classes, notably matching and scene geometry based, while the SI fusion techniques are classified into three classes, notably time, view and time-view driven. After reviewing the most relevant inter-view SI creation and SI fusion techniques guided by the proposed classification taxonomy, conclusions are drawn about the current status quo, thus allowing to better identify the next research challenges in the multiview WZ video coding paradigm.     

An optimal complexity H.264/AVC encoding for video streaming over next generation of wireless multimedia sensor networks

The next generation of sensor node such as Lotus mote enables higher performance, low power consumption, higher storage/memory and higher speed processing capability than the older generation, such as TelosB or MicaZ motes, which facilitates multimedia data pre-processing and compression in wireless multimedia sensor network (WMSN). In WMSNs, multimedia compression is the main issue that performed on a multimedia sensor node because the complexity of multimedia coding can cause a major energy consumption problem. This paper proposes an optimal complexity H.264 encoding for video streaming over next generation of WMSN. It develops the mathematical model for achieving optimal complexity encoding and proposed the most reliable H.264 encoder configuration setting on H.264 coding. The proposed mechanism ensures an optimal encoding complexity, less power consumption and visual quality of the multimedia. The experimental results of this research give an evidence to verify that the proposed mechanism guarantees high QoS performance.     

Advanced side information creation techniques and framework for Wyner–Ziv video coding

Recently, several distributed video coding (DVC) solutions based on the distributed source coding (DSC) paradigm have appeared in the literature. Wyner–Ziv (WZ) video coding, a particular case of DVC where side information is made available at the decoder, enable to achieve a flexible distribution of the computational complexity between the encoder and decoder, promising to fulfill novel requirements from applications such as video surveillance, sensor networks and mobile camera phones. The quality of the side information at the decoder has a critical role in determining the WZ video coding rate-distortion (RD) performance, notably to raise it to a level as close as possible to the RD performance of standard predictive video coding schemes. Towards this target, efficient motion search algorithms for powerful frame interpolation are much needed at the decoder. In this paper, the RD performance of a Wyner–Ziv video codec is improved by using novel, advanced motion compensated frame interpolation techniques to generate the side information. The development of these type of side information estimators is a difficult problem in WZ video coding, especially because the decoder only has available some reference, decoded frames. Based on the regularization of the motion field, novel side information creation techniques are proposed in this paper along with a new frame interpolation framework able to generate higher quality side information at the decoder. To illustrate the RD performance improvements, this novel side information creation framework has been integrated in a transform domain turbo coding based Wyner–Ziv video codec. Experimental results show that the novel side information creation solution leads to better RD performance than available state-of-the-art side information estimators, with improvements up to 2 dB; moreover, it allows outperforming H.264/AVC Intra by up to 3 dB with a lower encoding complexity.     

网内编码：无线视频传感网中分布式编码速率控制机制

Wyner-Ziv Video Coding (WZVC) is considered as a promising video coding scheme for Wireless Video Sensor Networks (WVSNs) due to its high compression efficiency and error resilience functionalities, as well as its low encoding complexity. To achieve a good Rate-Distortion (R-D) performance, the current WZVC paradigms usually adopt an end-to-end rate control scheme in which the decoder repeatedly requests the additional decoding data from the encoder for decoding Wyner-Ziv frames. Therefore, the waiting time of the additional decoding data is especially long in multihop WVSNs. In this paper, we propose a novel progressive in-network rate control scheme for WZVC. The proposed in-network puncturing-based rate control scheme transfers the partial channel codes puncturing task from the encoder to the relay nodes. Then, the decoder can request the additional decoding data from the relay nodes instead of the encoder, and the total waiting time for decoding Wyner-Ziv frames is reduced consequently. Simulation results validate the proposed rate control scheme.     

无反馈分布式视频编码中码率分配算法研究

码率分配是无反馈分布式视频编码中的关键技术之一.本文研究编码端快速边信息生成方法,基于边信息和Wyner-Ziv帧各个位平面误码率,提出了一种编码端码率分配算法(Bitplane Error Probability based Encoder Rate Control,BEP_ERC).还提出在编码端精确估计拉普拉斯-柯西混合分布(Laplace-Cauchy Mixture Distribution,LCMD)模型参数α和μ的思想.实验表明本文算法与现有算法相比,率失真性能提升0.1～0.4dB,且降低了编码端计算复杂度.     

An image feature-based method to efficiently determine inter-coding depth in HEVC

The quad-tree based picture partition scheme in High Efficiency Video Coding (HEVC) results in a more substantial increase in computational complexity than those incurred by its predecessor video coding standards because of the need in this scheme to determine the best coding unit (CU) partitions. In this paper, we propose a method to effectively reduce the computational complexity of inter-prediction coding in the HEVC standard. The relative displacement of the largest coding unit (LCU) at the corresponding position between adjacent frames is tested through optical flow (motion estimation). The texture intensity of the LCU at the given time is tested if the condition that determines the coding depth in advance cannot be satisfied. The depth of the coding unit (CU) can be determined in advance beyond the xCompressCU function by using our proposed method, which does not require the calculation of the rate-distortion (RD) cost for each level of depth, and thus reduces the circular traversal times of the xCompressCU function. Experimental results proved that our proposed method is effective, as it reduced the computational complexity of an encoder by 53.2% on average, and had a slight influence on coding performance.     

新一代基于HEVC的3D视频编码技术

HEVC标准出台后,新一代基于HEVC的多视点加深度编码也将正式推出。基于HEVC的3D视频编码作为HEVC标准的扩展部分,主要面向立体电视和自由立体视频。从该编码方式的基本结构出发,较全面地介绍了视频编码方式、深度图编码方式和对深度图的编码控制三个方面的关键技术,包括视点间运动预测、深度图建模模式和视点合成优化等技术。     

Hybrid model-and-object-based real-time conversational video coding

Bandwidth-constrained real-time conversational video communications (such as mobile teleconferencing) require video codecs with good rate-distortion characteristics at low bit-rates and modest computational complexity. While target-specific object-based and model-based coding methods have been proposed for low bit-rate conversational video coding, difficulties in generalization and high computational complexity hinder their practical utilization. In this paper, we propose a low bit-rate coding method for typical conversational video by combining two-dimensional model-based coding of face regions and object-based coding of non-face head-shoulder regions, achieving high-quality face reconstruction and low overall bit-rate with real-time encoding capability. Experiments on typical conversational test sequences confirm that, compared to other conversational video codecs, our model-and-object-based coding method offers superior rate-distortion performance at low bit-rates.     

Successively Structured Gaussian Two-terminal Source Coding

Multiterminal source coding refers to separate encoding and joint decoding of multiple correlated sources. Joint decoding requires all the messages to be decoded simultaneously which is exponentially more complex than a sequence of single-message decodings. Inspired by previous work on successive coding, we apply the successive Wyner-Ziv coding, which is inherently a low complexity approach of obtaining a prescribed distortion, to the two-terminal source coding scheme. First, we consider 1-helper problem where one source provides partial side information to the decoder to help the reconstruction of the main source. Our results show that the successive coding strategy is an optimal strategy in the sense of achieving the rate-distortion function. By developing connections between source encoding and data fusion steps, it is shown that the whole rate-distortion region for the 2-terminal source coding problem is achievable using the successive coding strategy. Comparing the performance of the sequential coding with the performance of the successive coding, we show that there is no sum-rate loss when the side information is not available at the encoder. This result is of special interest in some applications such as video coding where there are processing and storage constraints at the encoder. Finally, we provide an achievable rate-distortion region for the m-terminal source coding.

Deep chroma prediction of Wyner–Ziv frames in distributed video coding of wireless capsule endoscopy video

Compression of captured video frames is crucial for saving the power in wireless capsule endoscopy (WCE). A low complexity encoder is desired to limit the power consumption required for compressing the WCE video. Distributed video coding (DVC) technique is best suitable for designing a low complexity encoder. In this technique, frames captured in RGB colour space are converted into YCbCr colour space. Both Y and CbCr representing luma and chroma components of the Wyner–Ziv (WZ) frames are processed and encoded in existing DVC techniques proposed for WCE video compression. In the WCE video, consecutive frames exhibit more similarity in texture and colour properties. The proposed work uses these properties to present a method for processing and encoding only the luma component of a WZ frame. The chroma components of the WZ frame are predicted by an encoder–decoder based deep chroma prediction model at the decoder by matching luma and texture information of the keyframe and WZ frame. The proposed method reduces the computations required for encoding and transmitting of WZ chroma component. The results show that the proposed DVC with a deep chroma prediction model performs better when compared to motion JPEG and existing DVC systems for WCE at the reduced encoder complexity.     

Object-based stereo video compression using fractals and shape-adaptive DCT

Based on the classical fractal video compression method, an improved object-based stereo video compression scheme with Shape-Adaptive DCT is proposed in this paper. Firstly, we use more effective macroblock partition scheme instead of classical quadtree partition scheme; thus reducing the block searching strategy. The stereo fractal video coding is proposed which matches the macroblock with two reference frames in left and right view results in increasing compression ratio and reducing bit rate when transmitting compressed stereo data. The stereo codec combines the Motion Compensation Prediction (MCP) and Disparity Compensation Prediction (DCP). Fractal coding is adopted and each object is encoded independently by a prior video segmentation alpha plane, which is defined exactly as in MPEG-4. The testing results with the nature monocular and stereo video sequences provide promising performances at low bit rate coding. We believe it will be a powerful and efficient technique for the object-based monocular and stereo video sequences coding.