An automatic region-based video sequence codec based on fractal compression

In order to improve the performance of fractal video coding, we explore a novel fractal video sequences codec with automatic region-based functionality. To increase the quality of decoding image, intra frame coding, deblocking loop filter and sub-pixel block matching are applied to the codec. An efficient searching algorithm is used to increase the compression ratio and encoding speed. Automatic region-based fractal video sequences coding reduces coding stream greatly. Experimental results indicate that the proposed algorithm is more robust, and provides much less encoding time and bitrate while maintaining the quality of decompression image than the conventional CPM/NCIM method and other related references. We compare the proposed algorithm with three algorithms in Refs. [24], [25], [26], and the results of all these four algorithms are compared with H.264. The bitrate of the proposed algorithm is decreased by 0.11% and the other algorithms are increased by 4.29%, 6.85% and 11.62%, respectively. The average PSNR degradations of the four algorithms are 0.71 dB, 0.48 dB, 0.48 dB and 0.75 dB. So the bitrate of the proposed algorithm is decreased and the other algorithms are increased. At the meantime the compression time is reduced greatly, about 79.19% on average. The results indicate that, on average, the proposed automatic region-based fractal video sequences coding system can save compression time 48.97% and bitrate 52.02% with some image quality degradation in comparison with H.264, since they are all above 32 dB and the human eyes are insensitive to the differences.     

Multi-symbol QIM video watermarking

This paper introduces the theoretical framework allowing for the binary quantization index modulation (QIM) embedding techniques to be extended towards multiple-symbol QIM (m-QIM, where m stands for the number of symbols on which the mark is encoded prior to its embedding). The underlying detection method is optimized with respect to the minimization of the average error probability, under the hypothesis of white, additive Gaussian behavior for the attacks. This way, for prescribed transparency and robustness constraints, the data payload is increased by a factor of log₂m.m-QIM is experimentally validated under the frameworks of the MEDIEVALS French national project and of the SPY ITEA2 European project, related to MPEG-4 AVC robust and semi-fragile watermarking applications, respectively. The experiments are three-folded and consider the data payload–robustness–transparency tradeoff. In the former case, the main benefit is the increase of data payload by a factor of log₂m while keeping fixed robustness (variations lower than 3% of the bit error rate after additive noise, transcoding and Stirmark random bending attacks) and transparency (set to average PSNR=45 dB and 65 dB for SD and HD encoded content, respectively). The experiments consider 1 h of video content. In the semi-fragile watermarking case, the m-QIM main advantage is a relative gain factor of 0.11 of PSNR for fixed robustness (against transcoding), fragility (to content alteration) and the data payload. The experiments consider 1 h 20 min of video content.     

No-reference pixel based video quality assessment for HEVC decoded video

This paper proposes a No-Reference (NR) Video Quality Assessment (VQA) method for videos subject to the distortion given by the High Efficiency Video Coding (HEVC) scheme. The assessment is performed without access to the bitstream. The proposed analysis is based on the transform coefficients estimated from the decoded video pixels, which is used to estimate the level of quantization. The information from this analysis is exploited to assess the video quality. HEVC transform coefficients are modeled with a joint-Cauchy probability density function in the proposed method. To generate VQA features the quantization step used in the Intra coding is estimated. We map the obtained HEVC features using an Elastic Net to predict subjective video quality scores, Mean Opinion Scores (MOS). The performance is verified on a dataset consisting of HEVC coded 4 K UHD (resolution equal to 3840 × 2160) video sequences at different bitrates and spanning a wide range of content. The results show that the quality scores computed by the proposed method are highly correlated with the mean subjective assessments.     

Robust Video Transmission With Distributed Source Coded Auxiliary Channel

We propose a novel solution to the problem of robust, low-latency video transmission over lossy channels. Predictive video codecs, such as MPEG and H.26x, are very susceptible to prediction mismatch between encoder and decoder or “drift” when there are packet losses. These mismatches lead to a significant degradation in the decoded quality. To address this problem, we propose an auxiliary codec system that sends additional information alongside an MPEG or H.26x compressed video stream to correct for errors in decoded frames and mitigate drift. The proposed system is based on the principles of distributed source coding and uses the (possibly erroneous) MPEG/H.26x decoder reconstruction as side information at the auxiliary decoder. The distributed source coding framework depends upon knowing the statistical dependency (or correlation) between the source and the side information. We propose a recursive algorithm to analytically track the correlation between the original source frame and the erroneous MPEG/H.26x decoded frame. Finally, we propose a rate-distortion optimization scheme to allocate the rate used by the auxiliary encoder among the encoding blocks within a video frame. We implement the proposed system and present extensive simulation results that demonstrate significant gains in performance both visually and objectively (on the order of 2 dB in PSNR over forward error correction based solutions and 1.5 dB in PSNR over intrarefresh based solutions for typical scenarios) under tight latency constraints.      

Correlation based universal image/video coding loss recovery

Coding artifacts are annoying in highly compressed signals. Most of the existing artifact reduction methods are designed for one specific type of artifacts, codecs, and bitrates, which are complex and exclusive for one type of artifact reduction. Since both the compressed image/video and the coding error contain information of the original signal, they are highly correlated. Therefore, we try to recover some lost data based on the correlation between the compressed signal and the coding error, and introduce a novel and universal artifact reduction method. Firstly, according to the spatial correlation among pixels, a pixel-adaptive anisotropic filter is designed to reconstruct the distorted signal. Next, a globally optimal filter is designed to further recover the coding loss. Experimental results demonstrate that within an extensive range of bitrates, the proposed method achieves about 0.8 dB, 0.45 dB, 0.3 dB, and 0.2 dB on average of PSNR improvement for JPEG, MPEG4, H.264/AVC, and HEVC compressed signals, respectively.     

Exploiting quantization and spatial correlation in virtual-noise modeling for distributed video coding

Aiming for low-complexity encoding, video coders based on Wyner–Ziv theory are still unsuccessfully trying to match the performance of predictive video coders. One of the most important factors concerning the coding performance of distributed coders is modeling and estimating the correlation between the original video signal and its temporal prediction generated at the decoder.One of the problems of the state-of-the-art correlation estimators is that their performance is not consistent across a wide range of video content and different coding settings. To address this problem we have developed a correlation model able to adapt to changes in the content and the coding parameters by exploiting the spatial correlation of the video signal and the quantization distortion.In this paper we describe our model and present experiments showing that our model provides average bit rate gains of up to 12% and average PSNR gains of up to 0.5 dB when compared to the state-of-the-art models. The experiments suggest that the performance of distributed coders can be significantly improved by taking video content and coding parameters into account.     

Just noticeable distortion model and its applications in video coding

We explore a new perceptually-adaptive video coding (PVC) scheme for hybrid video compression, in order to achieve better perceptual coding quality and operational efficiency. A new just noticeable distortion (JND) estimator for color video is first devised in the image domain. How to efficiently integrate masking effects together is a key issue of JND modelling. We integrate spatial masking factors with the nonlinear additivity model for masking (NAMM). The JND estimator applies to all color components and accounts for the compound impact of luminance masking, texture masking and temporal masking. Extensive subjective viewing confirms that it is capable of determining a more accurate visibility threshold that is close to the actual JND bound in human eyes. Secondly, the image-domain JND profile is incorporated into hybrid video encoding via the JND-adaptive motion estimation and residue filtering process. The scheme works with any prevalent video coding standards and various motion estimation strategies. To demonstrate the effectiveness of the proposed scheme, it has been implemented in the MPEG-2 TM5 coder and demonstrated to achieve average improvement of over 18% in motion estimation efficiency, 0.6 dB in average peak signal-to perceptual-noise ratio (PSPNR) and most remarkably, 0.17 dB in the objective coding quality measure (PSNR) on average. Theoretical explanation is presented for the improvement on the objective coding quality measure. With the JND-based motion estimation and residue filtering process, hybrid video encoding can be more efficient and the use of bits is optimized for visual quality.     

MPEG-2视频编码流的码率变换技术的研究

比较分析了两种视频编码流的码率变换器结构的性能,提出了一种在图像内按宏块的复杂度分配目标码字,并直接控制交流ＤＣＴ系数编码比特数的码率控制算法。计算机模拟实验结果表明：由于ＭＰＥＧ－２标准中半像素精度运动补偿的非线性,单环结构的信噪比性能比起双环结构低大约０．１ｄＢ;本文提出的码率控制算法优于ＴＭ５的码率控制算法。     

MPEG—2视频解码的VHDL描述与验证

本文提出一种MPEG－2视频解码的硬件结构,并采用VHDL进行了描述。辚实现MPEG－2视频时的实时解码,本文针对时序控制、变长码解码、反量化、TDCT、运动补偿和输入输出控制等各部分都提出了相应的性能的电路结构。验证和仿真的结果表明：本文的设计可以完成相应的功能,能被用于实现MPEG－2MP@ML的实时解码芯片。     

Fuzzy quantization based bit transform for low bit-resolution motion estimation

This study proposes a novel fuzzy quantization based bit transform for low bit-resolution motion estimation. We formalize the procedure of bit resolution reduction by two successive steps, namely interval partitioning and interval mapping. The former is a many-to-one mapping which determines motion estimation performance, while the latter is a one-to-one mapping. To gain a reasonable interval partitioning, we propose a non-uniform quantization method to compute coarse thresholds. They are then refined by using a membership function to solve the mismatch of pixel values near threshold caused by camera noise, coding distortion, etc. Afterwards, we discuss that the sum of absolute difference (SAD) is one of the fast matching metrics suitable for low bit-resolution motion estimation in the sense of mean squared errors. A fuzzy quantization based low bit-resolution motion estimation algorithm is consequently proposed. Our algorithm not only can be directly employed in video codecs, but also be applied to other fast or complexity scalable motion estimation algorithms. Extensive experimental results show that the proposed algorithm can always achieve good motion estimation performances for video sequences with various characteristics. Compared with one-bit transform, multi-thresholding two-bit transform, and adaptive quantization based two-bit transform, our bit transform separately gains 0.98 dB, 0.42 dB, and 0.24 dB improvement in terms of average peak signal-to-noise ratio, with less computational cost as well.     

Rate-distortion optimization of scalable video codecs

In this paper joint optimization of layers in the layered video coding is investigated. Through theoretical analysis and simulations, it is shown that, due to higher interactions between the layers in a SNR scalable codec, this type of layering technique benefits most from joint optimization of the layers. A method for joint optimization is then proposed, and its compression efficiency is contrasted against the separate optimization and an optimized single layer coder. It is shown that, in joint optimization of SNR scalable coders when the quantization step size of the enhancement layer is larger than half the step size of the base layer, an additional improvement is gained by not sending the enhancement zero valued quantized coefficients, provided they are quantized at the base-layer. This will result in a non-standard bitstream syntax and as an alternative for standard syntax, one may skip the inter coded enhancement macroblocks. Through extensive tests it is shown that while separate optimization of SNR coders is inferior to single layer coder by more than 2 dB, with joint optimization this gap is reduced to 0.3–0.5 dB. We have shown that through joint optimization quality of the base layer video is also improved over the separate optimization. It is also shown that spatial scalability like SNR scalability does benefit from joint optimization, though not being able to exploit the relation between the quantizer step sizes. The amount of improvement depends on the interpolation artifacts of upsampled base-layer and the residual quantization distortion of this layer. Hence, the degree of improvement depends on image contents as well as the bit rate budget. Simulation results show that joint optimization of spatial scalable coders is about 0.5–1 dB inferior to the single layer optimized coder, where its separate optimization counterpart like SNR scalability is more than 2 dB worse.     

Dictionary learning based reconstruction for distributed compressed video sensing

Distributed compressed video sensing (DCVS) is a framework that integrates both compressed sensing and distributed video coding characteristics to achieve a low-complexity video coding. However, how to design an efficient reconstruction by leveraging more realistic signal models that go beyond simple sparsity is still an open challenge. In this paper, we propose a novel “undersampled” correlation noise model to describe compressively sampled video signals, and present a maximum-likelihood dictionary learning based reconstruction algorithm for DCVS, in which both the correlation and sparsity constraints are included in a new probabilistic model. Moreover, the signal recovery in our algorithm is performed during the process of dictionary learning, instead of being employed as an independent task. Experimental results show that our proposal compares favorably with other existing methods, with 0.1–3.5 dB improvements in the average PSNR, and a 2–9 dB gain for non-key frames when key frames are subsampled at an increased rate.     

LMM-based frame-level rate control for H.264/AVC high-definition video coding

Accurate distribution modeling for the DCT coefficients is greatly important for us to analyze the rate–distortion (R–D) behavior of video encoders. From the experiment, we observed that most of the existing models, paying more attention to the standard-definition (SD) videos, tend not to work well for high-definition (HD) videos. Motivated by this, in this paper, we address the statistical characteristics of DCT coefficients of HD videos coded by H.264/AVC. The contributions of this paper are threefold: first, Laplacian Mixture Model (LMM) is proposed to model the residues instead of using Laplacian or Cauchy distribution; second, the LMM-based analytic rate and distortion models are derived; third, building on the proposed rate and distortion models, a frame-level rate control algorithm is developed. Experimental results show that the proposed rate control method achieves a PSNR improvement of up to 0.85 dB compared with the rate control scheme adopted in the H.264 reference software [1]. Apart from the average visual quality improvement, the temporal visual quality fluctuation is reduced by 17%.     

Video fragment format classification using optimized discriminative subspace clustering

All video streams consist of highly compressed coded data. A video stream must be decoded to identify a video. It is impossible to decode and identify a video fragment without knowing the correct video format. Therefore, the first issue that must be addressed is classification of video formats. Although several methods exist for classifying file formats, a technology that specifically classifies the formats of video fragments has not been developed. In this paper, we present a novel approach to classify the formats of small fragments of video streams. Our classification procedure involves construction of high-dimensional feature vectors by combining synchronization patterns extracted from training fragments. The feature vectors are classified using optimized discriminative subspace clustering (ODiSC). The experimental results show a minimum classification error rate of 4.2%, and the precision of identification of the formats was greater than 91% for the four video formats whose fragment size was 256 KB.     

Motion Compensated Three-Dimensional Frequency Selective Extrapolation for improved error concealment in video communication

During transmission of video data over error-prone channels the risk of getting severe image distortions due to transmission errors is ubiquitous. To deal with image distortions at decoder side, error concealment is applied. This article presents Motion Compensated Three-Dimensional Frequency Selective Extrapolation, a novel spatio-temporal error concealment algorithm. The algorithm uses fractional-pel motion estimation and compensation as initial step, being followed by the generation of a model of the distorted signal. The model generation is conducted by an enhanced version of Three-Dimensional Frequency Selective Extrapolation, an existing error concealment algorithm. Compared to this existent algorithm, the proposed one yields an improvement in concealment quality of up to 1.64 dB PSNR. Altogether, the incorporation of motion compensation and the improved model generation extends the already high extrapolation quality of the underlying Frequency Selective Extrapolation, resulting in a gain of more than 3 dB compared to other well-known error concealment algorithms.     

AVI视频到MPEG-2视频转换的实现

文章介绍了AVI格式视频流和MPEG格式视频流的特点,以及MPEG-2压缩模型和算法,具体讨论了一种实用的AVI格式视频流经过MPEG-2压缩转换成MPEG格式视频流的实现方法.     

Prediction of MPEG-coded video source traffic using recurrent neural networks

Predicting traffic generated by multimedia sources is needed for effective dynamic bandwidth allocation and for multimedia quality-of-service (QoS) control strategies implemented at the network edges. The time-series representing frame or visual object plane (VOP) sizes of an MPEG-coded stream is extremely noisy, and it has very long-range time dependencies. This paper provides an approach for developing MPEG-coded real-time video traffic predictors for use in single-step (SS) and multistep (MS) prediction horizons. The designed SS predictor consists of one recurrent network for I-VOPs and two feedforward networks for P- and B-VOPs, respectively. These are used for single-frame-ahead prediction. A moving average of the frame or VOP sizes time-series is generated from the individual frame sizes and used for both SS and MS prediction. The resulting MS predictor is based on recurrent networks, and it is used to perform two-step-ahead and four-step-ahead prediction, corresponding to multistep prediction horizons of 1 and 2 s, respectively. All of the predictors are designed using a segment of a single MPEG-4 video stream, and they are tested for accuracy on complete video streams with a variety of quantization levels, coded with both MPEG-1 and MPEG-4. Comparisons with SS prediction results of MPEG-1 coded video traces from the recent literature are presented. No similar results are available for prediction of MPEG-4 coded video traces and for MS prediction. These are considered unique contributions of this research.     

Video super-resolution based on automatic key-frame selection and feature-guided variational optical flow

This paper proposes a new video super-resolution method based on feature-guided variational optical flow. The key-frames are automatically selected and super-resolved using a method based on sparse regression. To overcome the blocking artifacts and deal with the case of small structures with large displacement, an efficient method based on feature-guided variational optical flow is used to super-resolve the non-key-frames. Experimental results show that the proposed method outperforms the existing benchmark in terms of both subjective visual quality and objective peak signal-to-noise ratio (PSNR). The average PSNR improvement from the bi-cubic interpolation is 7.15 dB for four datasets. Thanks to the flexibility of designed automatic key-frame selection and the validness of feature-guided variational optical flow, the proposed method is applicable to various practical video sequences.     

Efficient reconstruction scheme for transform domain Wyner-Ziv video coding

To minimize the errors of the reconstructed values and improve the quality of decoded image,an efficient reconstruction scheme for transform domain Wyner-Ziv (WZ) video coding is proposed.The reconstruction scheme exploits temporal correlation of the coefficient bands,the WZ decoded bits stream and the side information efficiently.When side information is outside the decoded quantization bin,the reconstructed value is derived using expectation of the WZ decoded bit stream and the side information.When side information is within the decoded quantization bin,the reconstructed value is derived using the biased predictor.Simulation results show that the proposed reconstruction scheme gains up to 1.32 dB compared with the commonly used boundary reconstruction scheme at the same bit rates and similar computation cost.     

Rate control based on linear regression for H.264/MPEG-4 AVC

An efficient rate control algorithm is required to transmit coded video sequences without abrupt changes in quality due to the limited channel bandwidth. Thus, in this paper, an efficient rate control method is proposed for the H.264/MPEG (Moving Picture Experts Group)-4 Advanced Video Coding (AVC) baseline profile that meets the above constraint by using a ρ-domain source model. Firstly, a simple target bits determination method that considers the ideal buffer level is proposed, and secondly, a method of adequately determining the Quantization Parameter (QP) is presented using two kinds of linear regression. The experimental results show that the proposed rate control algorithm using the above two methods performs better than other rate control algorithms for H.264/MPEG-4 AVC in terms of the rate estimation error and the standard deviation of the Peak Signal-to- Noise Ratio (PSNR), which provides a measure of the constancy of the video quality throughout the whole video sequence.