Advanced side information creation techniques and framework for Wyner–Ziv video coding

Recently, several distributed video coding (DVC) solutions based on the distributed source coding (DSC) paradigm have appeared in the literature. Wyner–Ziv (WZ) video coding, a particular case of DVC where side information is made available at the decoder, enable to achieve a flexible distribution of the computational complexity between the encoder and decoder, promising to fulfill novel requirements from applications such as video surveillance, sensor networks and mobile camera phones. The quality of the side information at the decoder has a critical role in determining the WZ video coding rate-distortion (RD) performance, notably to raise it to a level as close as possible to the RD performance of standard predictive video coding schemes. Towards this target, efficient motion search algorithms for powerful frame interpolation are much needed at the decoder. In this paper, the RD performance of a Wyner–Ziv video codec is improved by using novel, advanced motion compensated frame interpolation techniques to generate the side information. The development of these type of side information estimators is a difficult problem in WZ video coding, especially because the decoder only has available some reference, decoded frames. Based on the regularization of the motion field, novel side information creation techniques are proposed in this paper along with a new frame interpolation framework able to generate higher quality side information at the decoder. To illustrate the RD performance improvements, this novel side information creation framework has been integrated in a transform domain turbo coding based Wyner–Ziv video codec. Experimental results show that the novel side information creation solution leads to better RD performance than available state-of-the-art side information estimators, with improvements up to 2 dB; moreover, it allows outperforming H.264/AVC Intra by up to 3 dB with a lower encoding complexity.     

Deep chroma prediction of Wyner–Ziv frames in distributed video coding of wireless capsule endoscopy video

Compression of captured video frames is crucial for saving the power in wireless capsule endoscopy (WCE). A low complexity encoder is desired to limit the power consumption required for compressing the WCE video. Distributed video coding (DVC) technique is best suitable for designing a low complexity encoder. In this technique, frames captured in RGB colour space are converted into YCbCr colour space. Both Y and CbCr representing luma and chroma components of the Wyner–Ziv (WZ) frames are processed and encoded in existing DVC techniques proposed for WCE video compression. In the WCE video, consecutive frames exhibit more similarity in texture and colour properties. The proposed work uses these properties to present a method for processing and encoding only the luma component of a WZ frame. The chroma components of the WZ frame are predicted by an encoder–decoder based deep chroma prediction model at the decoder by matching luma and texture information of the keyframe and WZ frame. The proposed method reduces the computations required for encoding and transmitting of WZ chroma component. The results show that the proposed DVC with a deep chroma prediction model performs better when compared to motion JPEG and existing DVC systems for WCE at the reduced encoder complexity.     

An improved side information generation for distributed video coding

This paper presents a side information (SI) scheme for distributed video coding based on multilayer perceptron. The suggested scheme predicts a Wyner–Ziv (WZ) frame from two decoded key frames. The network is trained offline using patterns from different standard video sequences with varied motion characteristics to achieve generalization. The proposed scheme is simulated along with other standard video coding schemes. Performance comparisons have been made with respect to training convergence, rate distortion (RD), peak signal to noise ratio (PSNR), number of requests per SI frame, decoding time requirement, etc. In general, it is observed that the proposed scheme has a superior SI frame generation capability as compared to its competent schemes.     

Adaptive Correlation Noise Model for DC Coefficients in Wyner‐Ziv Video Coding

An adaptive correlation noise model (CNM) construction algorithm is proposed in this paper to increase the efficiency of parity bits for correcting errors of the side information in transform domain Wyner‐Ziv (WZ) video coding. The proposed algorithm introduces two techniques to improve the accuracy of the CNM. First, it calculates the mean of direct current (DC) coefficients of the original WZ frame at the encoder and uses it to assist the decoder to calculate the CNM parameters. Second, by considering the statistical property of the transform domain correlation noise and the motion characteristic of the frame, the algorithm adaptively models the DC coefficients of the correlation noise with the Gaussian distribution for the low motion frames and the Laplacian distribution for the high motion frames, respectively. With these techniques, the proposed algorithm is able to make a more accurate approximation to the real distribution of the correlation noise at the expense of a very slight increment to the coding complexity. The simulation results show that the proposed algorithm can improve the average peak signal‐to‐noise ratio of the decoded WZ frames by 0.5 dB to 1.5 dB.     

无反馈分布式视频编码中码率分配算法研究

码率分配是无反馈分布式视频编码中的关键技术之一.本文研究编码端快速边信息生成方法,基于边信息和Wyner-Ziv帧各个位平面误码率,提出了一种编码端码率分配算法(Bitplane Error Probability based Encoder Rate Control,BEP_ERC).还提出在编码端精确估计拉普拉斯-柯西混合分布(Laplace-Cauchy Mixture Distribution,LCMD)模型参数α和μ的思想.实验表明本文算法与现有算法相比,率失真性能提升0.1～0.4dB,且降低了编码端计算复杂度.     

分布式压缩视频编码的帧分类重构方法研究

在很多的应用场景中需要具有低复杂度的视频编码器,新兴的分布式视频编码和压缩感知技术正好适用于这些场景中,因而出现了一种新的视频编码方案——分布式压缩视频编码。在现有的一些分布式压缩视频编码方案中,视频帧在编码端是独立编码,在解码端进行联合解码,具体来说就是关键帧独立解码,非关键帧在由关键帧生成的边信息的帮助下进行解码,这就忽略了非关键帧之间的相关性。本文提出一个新的分布式视频编码方案,将非关键帧分为主非关键帧和次非关键帧,主非关键帧利用关键帧生成地边信息进行解码,而次非关键帧先利用相邻的主非关键帧进行观测值预测,然后再利用关键帧生成的边信息进行解码。实验结果表明,在本文提出的框架下,非关键帧的重构质量提高了有2dB~4dB。      

Flexible distribution of complexity by hybrid predictive-distributed video coding

There is currently limited flexibility for distributing complexity in a video coding system. While rate-distortion-complexity (RDC) optimization techniques have been proposed for conventional predictive video coding with encoder-side motion estimation, they fail to offer true flexible distribution of complexity between encoder and decoder since the encoder is assumed to have always more computational resources available than the decoder. On the other hand, distributed video coding solutions with decoder-side motion estimation have been proposed, but hardly any RDC optimized systems have been developed.To offer more flexibility for video applications involving multi-tasking or battery-constrained devices, in this paper, we propose a codec combining predictive video coding concepts and techniques from distributed video coding and show the flexibility of this method in distributing complexity. We propose several modes to code frames, and provide complexity analysis illustrating encoder and decoder computational complexity for each mode. Rate distortion results for each mode indicate that the coding efficiency is similar. We describe a method to choose which mode to use for coding each inter frame, taking into account encoder and decoder complexity constraints, and illustrate how complexity is distributed more flexibly.     

Multiview side information creation for efficient Wyner–Ziv video coding: Classifying and reviewing

In recent years, the interest in multiview video systems has increased. In these systems, a typical predictive coding approach exploits the inter-view correlation at a joint encoder, requiring the various cameras to communicate among them. However, many applications ask for simple sensing systems preventing the various cameras to communicate among them, and thus the adoption of a predictive coding approach. Wyner–Ziv (WZ) video coding is a promising solution for those applications since it is the WZ decoder task to (fully or partly) exploit the video redundancy. The rate-distortion (RD) performance of WZ video coding strongly depends on the quality of the so-called side information (SI), which is a decoder estimate of the original frame to code. In multiview WZ (MV-WZ) video coding, the target is to exploit in the best way the available correlation not only in time, as for the monoview case, but also between views. Thus, the multiview SI results from the fusion of a temporally created SI and an inter-view created SI. In this context, the main objective of this paper is to propose a classification taxonomy to organize the many inter-view SI creation and SI fusion techniques available in the literature and to review the most relevant techniques in each class. The inter-view SI creation techniques are classified into two classes, notably matching and scene geometry based, while the SI fusion techniques are classified into three classes, notably time, view and time-view driven. After reviewing the most relevant inter-view SI creation and SI fusion techniques guided by the proposed classification taxonomy, conclusions are drawn about the current status quo, thus allowing to better identify the next research challenges in the multiview WZ video coding paradigm.     

Independent key frame coding using correlated pixels in distributed video coding

A novel intra-coding technique is proposed that eliminates the requirement of a secondary coding scheme for coding the key frames in distributed video coding (DVC). The proposed technique uses the Slepian-Wolf theorem and Wyner-Ziv (WZ) coding with spatially predicted information to transmit the key-frames to the DVC decoder. Simulation results show that the proposed WZ-intra coding technique (WZ-I) can achieve up to 5 dB PSNR gain compared to MPEG-2 intra coding (MPEG-I) at the same bit rate with negligible computational cost to the encoder     

Fast macroblock encoding algorithm based on rate-distortion activity for multiview video coding

Multiview video coding (MVC) is the appendix H of H.264/AVC, and it requires a great amount of time to compress multiple viewpoints׳ video with complex prediction structures. To reduce the whole computational complexity of MVC, this paper proposes a fast macroblock (MB) encoding algorithm based on rate-distortion (RD) activity, and it includes the fast mode decision and the fast motion/disparity estimation. First, the RD activity type of the current MB is calculated by utilizing the Skip/Direct RD cost and the average RD costs of classified MB modes. Then, through utilizing the RD activity type and RD costs of the estimated modes, the selection of candidate modes, the early decision of Skip/Direct mode, and the reduction of Inter8×8 mode estimation are all presented in the fast mode decision. By using the RD activity type and the correlations of vectors, the selection of search center and the prediction of search range are introduced in the fast motion/disparity estimation. In addition, the proposed algorithm can be applied to temporal and inter-view views as well as anchor and non-anchor frames. An experiment with a wide range of video scenes, camera setups and quantization parameters was implemented, and the results confirmed that the proposed algorithm can reduce the encoding time significantly while maintaining a similar RD performance as the original MVC encoder. Compared to the state-of-the-art algorithms, the proposed algorithm also demonstrated better performances in the various test cases.     

Side information generation with auto regressive model for low-delay distributed video coding

In this paper, we propose an auto regressive (AR) model to generate the high quality side information (SI) for Wyner-Ziv (WZ) frames in low-delay distributed video coding, where the future frames are not used for generating SI. In the proposed AR model, the SI of each pixel within the current WZ frame t is generated as a linear weighted summation of the pixels within a window in the previous reconstructed WZ/Key frame t − 1 along the motion trajectory. To obtain accurate SI, the AR model is used in both temporal directions in the reconstructed WZ/Key frames t − 1 and t − 2, and then the regression results are fused with traditional extrapolation result based on a probability model. In each temporal direction, a weighting coefficient set is computed by the least mean square method for each block in the current WZ frame t. In particular, due to the unavailability of future frames in low-delay distributed video coding, a centrosymmetric rearrangement is proposed for pixel generation in the backward direction. Various experimental results demonstrate that the proposed model is able to achieve a higher performance compared to the existing SI generation methods.     

Operational rate-distortion modeling for wavelet video coders

Based on our statistical investigation of a typical three-dimensional (3-D) wavelet codec, we present a unified mathematical model to describe its operational rate-distortion (RD) behavior. The quantization distortion of the reconstructed video frames is assessed by tracking the quantization noise along the 3-D wavelet decomposition trees. The coding bit-rate is estimated for a class of embedded video coders. Experimental results show that the model captures sequence characteristics accurately and reveals the relationship between wavelet decomposition levels and the overall RD performance. After being trained with offline RD data, the model enables accurate prediction of real RD performance of video codecs and therefore can enable optimal RD adaptation of the encoding parameters according to various network conditions.     

Improved side information generation algorithm for Wyner-Ziv video coding简

Side information has a significant influence on the rate-distortion（RD） performance of distributed video coding（DVC）. In the conventional motion compensated frame interpolation scheme, all blocks adopt the same side-information generation method regardless of the motion intensity inequality at different regions. In this paper, an improved method is proposed. The image blocks are classified into two modes, fast motion and slow motion, by simply computing the discrete cosine transformation（DCT） coefficients at the encoder. On the decoder, it chooses the direct interpolation and refined motion compensated interpolation correspondingly to generate side information. Experimental results show that the proposed method, without increasing the encoder complexity, can increase the average peak signal-to-noise ratio（PSNR） by up to 1~ 2 dB compared with the existing algorithm. Meanwhile, the proposed algorithm significantly improves the subjective quality of the side information.     

Perceptual-based distributed video coding

In this paper, we propose a perceptual-based distributed video coding (DVC) technique. Unlike traditional video codecs, DVC applies video prediction process at the decoder side using previously received frames. The predicted video frames (i.e., side information) contain prediction errors. The encoder then transmits error-correcting parity bits to the decoder to reconstruct the video frames from side information. However, channel codes based on i.i.d. noise models are not always efficient in correcting video prediction errors. In addition, some of the prediction errors do not cause perceptible visual distortions. From perceptual coding point of view, there is no need to correct such errors. This paper proposes a scheme for the decoder to perform perceptual quality analysis on the predicted side information. The decoder only requests parity bits to correct visually sensitive errors. More importantly, with the proposed technique, key frames can be encoded at higher rates while still maintaining consistent visual quality across the video sequence. As a result, even the objective PSNR measure of the decoded video sequence will increase too. Experimental results show that the proposed technique improves the R-D performance of a transform domain DVC codec both subjectively and objectively. Comparisons with a well-known DVC codec show that the proposed perceptual-based DVC coding scheme is very promising for distributed video coding framework.     

Distributed video coding based on vector quantization: Application to capsule endoscopy

We present in this paper a new distributed video coding (DVC) architecture for wireless capsule endoscopy. It is based on the state of the art DVC systems, but without using key frames. Instead, it uses an adapted vector quantization (VQ) with a searching complexity that is shifted to the decoder. VQ allows creating a good side information (SI) by exploiting the similarities in human anatomy. Thus, SI is created from a codebook (CB) rather than by motion compensated prediction. This approach decreases largely the complexity of the encoder, which codes only Wyner-Ziv frames, and allows a progressive decoding. The encoder of the proposed DVC generates only a simple hash that is used by the decoder to select the corresponding VQ codeword. The obtained experimental results show that rate-distortion results are better than those of JPEG, and show the possibility of using scalable coding to control the used rate and energy.     

Tracking a dynamic set of feature points

We address the problems of tracking a set of feature points over a long sequence of monocular images as well as how to include and track new feature points detected in successive frames. Due to the 3-D movement of the camera, different parts of the images exhibit different image motion. Tracking discrete features can therefore be decomposed into several independent and local problems. Accordingly, we propose a localized feature tracking algorithm. The trajectory of each feature point is described by a 2-D kinematic model. Then to track a feature point, an interframe motion estimation scheme is designed to obtain the estimates of interframe motion parameters. Subsequently, using the estimates of motion parameters, corresponding points are identified to subpixel accuracy. Afterwards, the temporal information is processed to facilitate the tracking scheme. Since different feature points are tracked independently, the algorithm is able to handle the image motion arising from general 3-D camera movements. On the other hand, in addition to tracking feature points detected at the beginning, an efficient way to dynamically include new points extracted in subsequent frames is devised so that the information in a sequence is preserved. Experimental results for several image sequences are also reported.     

Motion Compensated Subband Video Coding with Arbitrarily Shaped Region Adaptivity

The performance of Motion Compensated Discrete Cosine Transform (MC‐DCT) video coding is improved by using the region adaptive subband image coding [18]. On the assumption that the video is acquired from the camera on a moving platform and the distance between the camera and the scene is large enough, both the motion of camera and the motion of moving objects in a frame are compensated. For the compensation of camera motion, a feature matching algorithm is employed. Several feature points extracted using a Sobel operator are used to compensate the camera motion of translation, rotation, and zoom. The illumination change between frames is also compensated. Motion compensated frame differences are divided into three regions called stationary background, moving objects, and newly emerging areas each of which is arbitrarily shaped. Different quantizers are used for different regions. Compared to the conventional MC‐DCT video coding using block matching algorithm, our video coding scheme shows about 1.0‐dB improvements on average for the experimental video samples.     

一种空间域Wyner-Ziv视频编码系统的性能改进算法

分布式视频编码是建立在Slepian-Wolf和Wyner-Ziv信息编码理论基础上的全新视频编码框架,具有编码复杂度低,编码效率较高,抗误码性能好的特点.本文首先简单介绍了一种典型的分布式视频编码实现方案——空间域Wyner-Ziv视频编码,随后提出一种空间域Wyner-Ziv视频编码系统的性能改进算法,该算法在不增加编码复杂度的基础上,在解码端利用双向运动估计预测获取更高质量的边信息,同时采用基于Huber-Markov随机场约束的联合迭代解码算法重建图像.实验结果表明,在相同的输出码流情况下,本文改进算法在解码端重建图像的峰值信噪比与空间域Wyner-Ziv视频编码算法相比平均提高2dB,并且主观效果有所改善.     

Side information refinement using motion estimation in dc domain for transform-based distributed video coding

The key to good decoding performance in distributed video coding systems lies in the efficient prediction of frames using side information. Previously, this process has been performed without considering the progressive nature of the synthesis of the final frame. Observing the fact that incorrectly predicted areas of the current frame can be detected at different levels of final frame synthesis, an algorithm is proposed for refinement of side information synthesis based on an additional stage of motion estimation. The additional stage is performed on correctly decoded DC frames and is used to significantly improve the motion prediction of the final resolution frames, which leads to enhanced performance of the overall system.     

Low-complexity video coding via power-rate-distortion optimization

Wireless multimedia sensor networks (WMSNs) have been potentially applicable for several emerging applications. The resources, i.e., power and bandwidth available to visual sensors in a WMSN are, however, very limited. Hence, it is important but challenging to achieve efficient resource allocation and optimal video data compression while maximizing the overall network lifetime. In this paper, a power-rate-distortion (PRD) optimized resource-scalable low-complexity multiview video encoding scheme is proposed. In our video encoder, both the temporal and interview information can be exploited based on the comparisons of extracted media hashes without performing motion and disparity estimations, which are known to be time-consuming. We present a PRD model to characterize the relationship between the available resources and the RD performance of our encoder. More specifically, an RD function in terms of the percentages for different coding modes of blocks and the target bit rate under the available resource constraints is derived for optimal coding mode decision. The major goal here is to design a PRD model to optimize a “motion estimation-free” low-complexity video encoder for applications with resource-limited devices, instead of designing a general-purpose video codec to compete compression performance against current compression standards (e.g., H.264/AVC). Analytic results verify the accuracy of our PRD model, which can provide a theoretical guideline for performance optimization under limited resource constraints. Simulation results on joint RD performance and power consumption (measured in terms of encoding time) demonstrate the applicability of our video coding scheme for WMSNs.