Multiresolution Vector Quantization for Video Coding

In this work, we propose a coding technique that is based on the generalized block prediction of the multiresolution subband decomposition of motion compensated difference image frames. A segmentation mask is used to distinguish between the regions where motion compensation was effective and those regions where the motion model did not succeed. The difference image is decomposed into a multiresolution pyramid of subbands where the highest resolution subbands are divided into two regions, based on the information given by the segmentation mask. Only the coefficients of the regions corresponding to the motion model failure are considered in the highest resolution subbands. The remaining coefficients are coded using a multiresolution vector quantization scheme that exploits inter-band non-linear redundancy. In particular, blocks in one subimage are predicted from blocks of the adjacent lower resolution subimage with the same orientation. This set of blocks plays the role of a codebook built from coefficients inside the subband decomposition itself. Whenever the inter-band prediction does not give satisfactory results with respect to a target quality, the block coefficients are quantized using a lattice vector quantizer for a Laplacian source.     

A Motion Picture Coding Algorithm Using Adaptive DCT Encoding Based on Coefficient Power Distribution Classification

A motion picture coding algorithm using motion-compensated interframe prediction and the adaptive discrete cosine transform (DCT) encoding technique is proposed. High coding efficiency is obtained by the adaptive DCT encoding technique in which encoding parameters are fitted to widely varying characteristics of the interframe differential signal. Segmented DCT subblocks of interframe prediction error are classified into categories based on their coefficient power distribution characteristics. The adaptation gain results from using a suitable variable word length code set designated by the above classification for encoding each quantization index of DCT coefficients. In addition, a new coding parameter control method is introduced based on the information rate estimation of the current frame. This classification promotes high stability because good estimation accuracy of bits consumption for each DCT subblock is obtained by utilizing the category indexes. Simulation results show that the proposed algorithm has enough coding efficiency to transmit videoconferencing motion pictures through a 384 kbit/s channel.     

Edge-based motion compensated classified DCT with quadtree for image sequence coding

The motion compensated discrete cosine transform coding (MCDCT) is an efficient image sequence coding technique. In order to further reduce the bit-rate for the quantizied DCT coefficients and keep the visual quality, we propose an adaptive edge-based quadtree motion compensated discrete cosine transform coding (EQDCT). In our proposed algorithm, the overhead moving information is encoded by a quadtree structure and the nonedge blocks will be encoded at lower bit-rate but the edge blocks will be encoded at higher bit-rate. The edge blocks will be further classified into four different classes according to the orientations and locations of the edges. Each class of edge blocks selects the different set of the DCT coefficients to be encoded. By this method, we can just preserve and encode a few DCT coefficients, but still maintain the visual quality of the images. In the proposed EQDCT image sequence coding scheme, the average bit-rate of each frame is reduced to 0.072 bit/pixel and the average PSNR value is 32.11 dB.     

Temporal image sequence prediction using motion field interpolation

A new method for motion-compensated temporal prediction of image sequences is proposed. Motion vector fields in natural scenes should possess two basic properties. First, the field should be smoothly varying within moving objects to compensate for nonrigid or rotational motion, and scaling of objects. Second, the field should be discontinuous along the boundaries of the objects. In the proposed method the motion vector field is modelled using finite element methods and interpolated using adaptive interpolators to satisfy the above-stated requirements. This is particularly important when only very sparse estimates of motion vector fields are available in the decoder due to bit-rate constraints limiting the amount of overhead information that can be transmitted. The proposed prediction method can be applied for low-bit-rate video coding in conventional codecs based on motion-compensated prediction and transform coding, as well as in model-based codecs. The performance of the proposed method is compared with standard motion-compensated prediction based on block matching. It is shown that for simple video telephony scenes a reduction of more than 30% in the energy of the prediction error can be achieved with an unchanged number of transmitted motion vectors and with only a modest increase in computational complexity. When implemented in an H.261 codec the new prediction method can improve the peak SNR 1–2 dB producing a significant visual improvement.     

Multispectral code excited linear prediction coding and itsapplication in magnetic resonance images

This paper reports a multispectral code excited linear prediction (MCELP) method for the compression of multispectral images. Different linear prediction models and adaptation schemes have been compared. The method that uses a forward adaptive autoregressive (AR) model has been proven to achieve a good compromise between performance, complexity, and robustness. This approach is referred to as the MFCELP method. Given a set of multispectral images, the linear predictive coefficients are updated over nonoverlapping three-dimensional (3-D) macroblocks. Each macroblock is further divided into several 3-D micro-blocks, and the best excitation signal for each microblock is determined through an analysis-by-synthesis procedure. The MFCELP method has been applied to multispectral magnetic resonance (MR) images. To satisfy the high quality requirement for medical images, the error between the original image set and the synthesized one is further specified using a vector quantizer. This method has been applied to images from 26 clinical MR neuro studies (20 slices/study, three spectral bands/slice, 256x256 pixels/band, 12 b/pixel). The MFCELP method provides a significant visual improvement over the discrete cosine transform (DCT) based Joint Photographers Expert Group (JPEG) method, the wavelet transform based embedded zero-tree wavelet (EZW) coding method, and the vector tree (VT) coding method, as well as the multispectral segmented autoregressive moving average (MSARMA) method we developed previously.     

3D Color Set Partitioning in Hierarchical Trees

This paper introduces the 3D color set partitioning in hierarchical trees (3D-CSPIHT) low bit rate embedded video coding scheme. The codec exploits the correlation between temporal and spatial wavelet coefficients and the interdependency between luminance and chrominance components to code color video sequences without the need for explicit bit allocation. Besides offering rate scalability, the new codec also produces multi-resolution scalable code streams. The hierarchical variable size block matching motion estimation technique is also integrated to demonstrate the motion estimation option with 3D-CSPIHT. The coding results show that 3D-CSPIHT produces better performance and visual quality compared to 3D-SPIHT.     

一种新的小波可视电话编码算法

针对极低比特率应用提出一种新的结合H.263与SLCCA的混合小波视频编码算法。在提出的算法中,首先,用基于H.263的微调运动估计减小时间冗余,用无遗漏覆盖块运动补偿保证运动补偿误差帧的连续性;第二,对运动补偿误差帧进行小波变换得到全局能量压缩;第三,用SLCCA组织和表示小波变换后的数据,最后,运动向量的水平和垂直分量分别用自适应算法编码,算法在A级测试序列Akiyo和B级测试序列Foreman（QFIF）上测试取得了良好效果。     

Fixed and Adaptive Predictors for Hybrid Predictive/Transform Coding

Hybrid predictive/transform coding is studied. The usual formulation is to first apply a unitary transform and then code the transform coefficients with independent DPCM coders, i.e., the prediction is performed in the transform domain. This structure is compared to spatial domain prediction, where a difference signal is formed in the spatial domain and then coded by a transform coder. A linear spatial domain predictor which minimizes the mean square prediction error also minimizes the mean square of each transform coefficient. The two structures are equivalent if the transform domain prediction scheme is extended to a more general predictor. Hence, the structure that gives the easiest implementation can be chosen. The spatial domain structure is preferred for motion compensation and for line interlaced video signals. Interframe hybrid coding experiments are performed on interlaced videophone scenes using an adaptive transform coder. Motion compensation gives a rate reduction of 25-35 percent compared to frame difference prediction with the same mean square error. The subjective advantage is even greater, since the "dirty window" effect is not present with motion compensation. It is important to perform the motion estimation with fractional pel accuracy. Field coding with a switched predictor using previous field in moving areas is an interesting alternative to frame coding with frame difference prediction.     

An efficient motion vector coding scheme based on minimum bitrateprediction

Motion vector coding efficiency is becoming an important issue in low bitrate video coding because of its increasing relative bit portion. This work presents a new motion vector coding technique based on minimum bitrate prediction. In the proposed scheme, a predicted motion vector is chosen from the three causal neighboring motion vectors so that it can produce a minimum bitrate in motion vector difference coding. Then the prediction error, or motion vector difference (MVD), and the mode information (MODE) for determining the predicted motion vector at a decoder are coded and transmitted in order. Sending bits for the MVD ahead of bits for the MODE, the scheme can minimize the bit amount for the MODE by taking advantage of the fact that the minimum bitrate predictor is used for motion vector prediction. Adaptively combining this minimum bitrate prediction scheme with the conventional model-based prediction scheme, more efficient motion vector coding can be achieved. The proposed scheme improves the coding efficiency noticeably for various video sequences.     

A 64 kbit/s Videophone codec with forward analysis and control

A new combination of coding methods for a 64 kbit/s transmission system for typical videophone situations is investigated. The codec structure is based on a standard hybrid discrete cosine transform (DCT) codec with temporal prediction. The picture is divided blockwise into changed and unchanged areas. One motion vector with subpel accuracy is computed and transmitted for each block of the changed area. For the forward analysis, the prediction error is calculated in the whole picture. Only the blocks with the highest prediction errors are updated by a DCT with a perception adaptive quantization. The number of DCT update blocks depends on the remaining bits after the transmission of the overhead information. The codec is controlled by a forward analysis of the prediction error and is not based on a buffer control. The spatial resolution of the source signal is reduced in two steps to prevent a codec overload caused by too much activity between two frames.     

Suboptimality of the Karhunen-Loeve transform for transform coding

We examine the performance of the Karhunen-Loeve transform (KLT) for transform coding applications. The KLT has long been viewed as the best available block transform for a system that orthogonally transforms a vector source, scalar quantizes the components of the transformed vector using optimal bit allocation, and then inverse transforms the vector. This paper treats fixed-rate and variable-rate transform codes of non-Gaussian sources. The fixed-rate approach uses an optimal fixed-rate scalar quantizer to describe the transform coefficients; the variable-rate approach uses a uniform scalar quantizer followed by an optimal entropy code, and each quantized component is encoded separately. Earlier work shows that for the variable-rate case there exist sources on which the KLT is not unique and the optimal quantization and coding stage matched to a "worst" KLT yields performance as much as 1.5 dB worse than the optimal quantization and coding stage matched to a "best" KLT. In this paper, we strengthen that result to show that in both the fixed-rate and the variable-rate coding frameworks there exist sources for which the performance penalty for using a "worst" KLT can be made arbitrarily large. Further, we demonstrate in both frameworks that there exist sources for which even a best KLT gives suboptimal performance. Finally, we show that even for vector sources where the KLT yields independent coefficients, the KLT can be suboptimal for fixed-rate coding.     

Additive vector decoding of transform coded images

In a standard transform coding scheme of images or video, the decoder can be implemented by a table-lookup technique without the explicit use of an inverse transformation, In this new decoding method, each received code index of a transform coefficient addresses a particular codebook to fetch a component code vector that resembles the basis vector of the linear transformation. The output image is then reconstructed by summing a small number of nonzero component code vectors. With a set of well-designed codebooks, this new decoder can exploit the correlation among the quantized transform coefficients to achieve better rate-distortion performance than the conventional decoding method. An iterative algorithm for designing a set of locally optimal codebooks from a training set of images is presented. We demonstrate that this new idea can be applied to decode improved quality pictures from the bitstream generated from a standard encoding scheme of still images or video, while the complexity is low enough to justify practical implementation.     

WHT-based composite motion compensated NTSC interframe directcoding

The motion compensated interframe differential pulse code modulation (DPCM) and discrete cosine transform (DCT) hybrid (MC DCT) coding was nominated as a standard scheme for component TV signals by ISO and ITU-R. However, in cases where an NTSC composite TV signal is used such as the United States and Japan, applying the MC DCT scheme with its luminance/chrominance separating and composing process causes unavoidable quality degradation. The reason for this additional process required for MC DCT is that a composite TV signal presents a “color subcarrier phase shift problem” in which the color subcarrier phase varies between a coding block and reference block according to the motion vector. In this paper, we propose a Walsh Hadamard transform (WHT)-based composite motion compensated NTSC interframe direct coding scheme. In this scheme, phase shifts of a color subcarrier and modulated chrominance components between a coding block and reference block can be effectively compensated by a simple process of coefficient permutation and polarity changes of several pairs of WHT coefficients to which 100% of the subcarrier energy and most of the modulated chrominance component's energy are packed. In the motion compensated DCT scheme, however, the energy of the color subcarrier and modulated chrominance components are spread over too many coefficients and a pair-based coefficient handling rule is not given to solve this problem. This paper demonstrates that the proposed scheme provides higher coding performance for a composite NTSC signal than does the motion compensated DCT scheme with its luminance/chrominance separating and composing process     

Video coding scheme using DCT-pyramid vector quantization

A new and effective video coding scheme for contribution quality is proposed. The CMTT/2, a joint committee of CCIR and CCITT, has proposed a video coding scheme (already approved at European level by ETS) working at 34-45 Mbit/s. Basically this proposal includes a DCT transform for spatial correlation removal and motion compensation for temporal correlation removal. The individual transform coefficients are then scalar quantized with a non uniform bit assignment. Starting from the CMTT/2 proposal, the study presents a new video coding scheme designed using a vector quantizer solution instead of the scalar one. Specifically, the pyramid vector quantization (PVQ) has been chosen as the vector quantization method as it is able to reduce the DCT coefficients Laplacian distribution. Simulation results show that the proposed video coding scheme gives the same contribution quality at 22 Mbit/s as the one obtained with the CMTT/2 proposal at 45 Mbit/s.     

Improved decoder for transform coding with application to the JPEGbaseline system

Transform coding, a simple yet efficient image coding technique, has been adopted by the Joint Photographic Experts Group (JPEG) as the basis for an emerging coding standard for compression of still images. However, for any given transform encoder, the conventional inverse transform decoder is suboptimal. Better performance can be obtained by a nonlinear interpolative decoder that performs table lookups to reconstruct the image blocks from the code indexes. Each received code index of an image block addresses a particular codebook to fetch a component vector. The image block can be reconstructed as the sum of the component vectors for that block. An iterative algorithm for designing a set of locally optimal codebooks is developed. Computer simulation results demonstrate that this improved decoding technique can be applied in the JPEG baseline system to decode enhanced quality pictures from the bit stream generated by the standard encoding scheme     

Motion-compensated wavelet transform coder for very low bit-rate visual telephony

This paper proposes a new motion-compensated wavelet transform video coder for very low bit-rate visual telephony. The proposed coder sequentially employs: (1) selective motion estimation on the wavelet transform domain, (2) motion-compensated prediction (MCP) of wavelet coefficients, and (3) selective entropy-constrained vector quantization (ECVQ) of the resultant MCP errors. The selective schemes in motion estimation and in quantization, which efficiently exploit the characteristic of image sequences in a visual telephony, considerably reduce the computational burden. The coder also employs a tree structure encoding to represent efficiently which blocks were encoded. In addition, in order to reduce the number of ECVQ codebooks and the image dependency of their performance, we introduce a preprocessing of signals which normalizes input vectors of ECVQ. Simulation results show that our video coder provides good PSNR (peak-to-peak signal-to-noise ratio) performance and efficient rate control.     

Moving object detection in the H.264/AVC compressed domain for video surveillance applications

In this paper a novel method is presented to detect moving objects in H.264/AVC [T. Wiegand, G. Sullivan, G. Bjontegaard, G. Luthra, Overview of the H.264/AVC video coding standard, IEEE Transactions on Circuits and Systems for Video Technology, 13 (7) (2003) 560–576] compressed video surveillance sequences. Related work, within the H.264/AVC compressed domain, analyses the motion vector field to find moving objects. However, motion vectors are created from a coding perspective and additional complexity is needed to clean the noisy field. Hence, an alternative approach is presented here, based on the size (in bits) of the blocks and transform coefficients used within the video stream. The system is restricted to the syntax level and achieves high execution speeds, up to 20 times faster than the related work. To show the good detection results, a detailed comparison with related work is presented for different challenging video sequences. Finally, the influence of different encoder settings is investigated to show the robustness of our system.     

Adaptive block transform coding of speech based on LPC vectorquantization

The authors describe several adaptive block transform speech coding systems based on vector quantization of linear predictive coding (LPC) parameters. Specifically, the authors vector quantize the LPC parameters (LPCVQ) associated with each speech block and transmit the index of the code vector as overhead information. This code vector will determine the short-term spectrum of the block and, in turn, can be used for optimal bit allocation among the transform coefficients. In order to get a better estimate of the speech spectrum, the authors also consider the possibility of incorporating pitch information in the coder. In addition, entropy-coded zero-memory quantization of the transform coefficients is considered as an alternative to Lloyd-Max quantization. An adaptive BTC scheme based on LPCVQ and using entropy-coded quantizers is developed. Extensive simulations are used to evaluate the performance of this scheme     

Edge oriented block motion estimation for video coding

Intensity-based block motion estimation and compensation algorithms are widely used to exploit temporal redundancies in video coding, although they suffer from several drawbacks. One of the problems is that blocks located on boundaries of moving objects are not estimated accurately. It causes poor motion-compensated prediction along the moving edges to which the human visual system is very sensitive. By considering the characteristics of block motions for typical image sequences, an intelligent classifier is proposed to separate blocks containing moving edges to improve on conventional intensity-based block matching approaches. The motion vectors of these blocks are computed using edge matching techniques, so that the motion-compensated frames are tied more closely to the physical features. The proposed method can then make use of this accurate motion information for edge blocks to compute the remaining non-edged blocks. Consequently, a fast and efficient block motion estimation algorithm is developed. Experimental results show that this approach gives a significant improvement in accuracy for motion-compensated frames and computational complexity, in comparison with the traditional intensity-based block motion estimation methods     

A very low bit rate video coder based on vector quantization

Describes a video coder based on a hybrid DPCM-vector quantization algorithm that is suited for bit rates ranging from 8-16 kb/s. The proposed approach involves segmenting difference images into variable-size and variable-shape blocks and performing segmentation and motion compensation simultaneously. The purpose of obtaining motion vectors for variable-size and variable-shape blocks is to improve the quality of motion estimation, particularly in those areas where the edges of moving objects are situated. For the larger blocks, decimation takes place in order to simplify vector quantization. For very active blocks, which are always of small dimension, a specific vector quantizer has been applied, the fuzzy classified vector quantizer (FCVQ). The coding algorithm described displays good performance in the compression of test sequences at the rates of 8 and 16 kb/s; the signal-to-noise ratios obtained are good in both cases. The complexity of the coder implementation is comparable to that of conventional hybrid coders, while the decoder is much simpler in this proposal.