Fine-coarse vector quantization

A fast method for searching an unstructured vector quantization (VQ) codebook is introduced and analyzed. The method, fine-coarse vector quantization (FCVQ), operates in two stages: a `fine' structured VQ followed by a table lookup `coarse' unstructured VQ. Its rate, distortion, arithmetic complexity, and storage are investigated using analytical and experimental means. Optimality condition and an optimizing algorithm are presented. The results of experiments with both uniform scalar quantization and tree-structured VQ (TSVQ) as the first stage are reported. Comparisons are made with other fast approaches to vector quantization, especially TSVQ. It is found that when rate, distortion, arithmetic complexity, and storage are all taken into account, FCVQ outperforms TSVQ in a number of cases. In comparison to full search quantization, FCVQ has much lower arithmetic complexity, at the expense of a slight increase in distortion and a substantial increase in storage. The increase in mean-squared error (over full search) decays as a negative power of the available storage     

An entropy-coded lattice vector quantizer for transform and subbandimage coding

A lattice-based vector quantizer (VQ) and noiseless code are proposed for transform and subband image coding. The quantization is simple to implement, and no vector codebooks need to be stored. The noiseless code enumerates lattice codevectors based on their (weighted) l(1) norm. A software implementation is able to handle lattice codebooks of size 2(256). The image coding performance is shown to be comparable or superior to the best encoding methods reported in the literature.     

High-resolution source coding for non-difference distortionmeasures: multidimensional companding

Entropy-coded vector quantization is studied using high-resolution multidimensional companding over a class of nondifference distortion measures. For distortion measures which are “locally quadratic” a rigorous derivation of the asymptotic distortion and entropy-coded rate of multidimensional companders is given along with conditions for the optimal choice of the compressor function. This optimum compressor, when it exists, depends on the distortion measure but not on the source distribution. The rate-distortion performance of the companding scheme is studied using an asymptotic expression for the rate-distortion function which parallels the Shannon lower bound for difference distortion measures. It is proved that the high-resolution performance of the scheme is arbitrarily close to the rate-distortion limit for large quantizer dimensions if the compressor function and the lattice quantizer used in the companding scheme are optimal, extending an analogous statement for entropy-coded lattice quantization and MSE distortion. The companding approach is applied to obtain a high-resolution quantizing scheme for noisy sources     

Binary lattice vector quantization with linear block codes andaffine index assignments

We determine analytic expressions for the performance of some low-complexity combined source-channel coding systems. The main tool used is the Hadamard transform. In particular, we obtain formulas for the average distortion of binary lattice vector quantization with affine index assignments, linear block channel coding, and a binary-symmetric channel. The distortion formulas are specialized to nonredundant channel codes for a binary-symmetric channel, and then extended to affine index assignments on a binary-asymmetric channel. Various structured index assignments are compared. Our analytic formulas provide a computationally efficient method for determining the performance of various coding schemes. One interesting result shown is that for a uniform source and uniform quantizer, the natural binary code is never optimal for a nonsymmetric channel, even though it is known to be optimal for a symmetric channel     

On lattice quantization noise

We present several results regarding the properties of a random vector, uniformly distributed over a lattice cell. This random vector is the quantization noise of a lattice quantizer at high resolution, or the noise of a dithered lattice quantizer at all distortion levels. We find that for the optimal lattice quantizers this noise is wide-sense-stationary and white. Any desirable noise spectra may be realized by an appropriate linear transformation (“shaping”) of a lattice quantizer. As the dimension increases, the normalized second moment of the optimal lattice quantizer goes to 1/2πe, and consequently the quantization noise approaches a white Gaussian process in the divergence sense. In entropy-coded dithered quantization, which can be modeled accurately as passing the source through an additive noise channel, this limit behavior implies that for large lattice dimension both the error and the bit rate approach the error and the information rate of an additive white Gaussian noise (AWGN) channel     

Asymptotic distribution of the errors in scalar and vectorquantizers

High-rate (or asymptotic) quantization theory has found formulas for the average squared length (more generally, the qth moment of the length) of the error produced by various scalar and vector quantizers with many quantization points. In contrast, this paper finds an asymptotic formula for the probability density of the length of the error and, in certain special cases, for the probability density of the multidimensional error vector, itself. The latter can be used to analyze the distortion of two-stage vector quantization. The former permits one to learn about the point density and cell shapes of a quantizer from a histogram of quantization error lengths. Histograms of the error lengths in simulations agree well with the derived formulas. Also presented are a number of properties of the error density, including the relationship between the error density, the point density, and the cell shapes, the fact that its qth moment equals Bennett's integral (a formula for the average distortion of a scalar or vector quantizer), and the fact that for stationary sources, the marginals of the multidimensional error density of an optimal vector quantizer with large dimension are approximately i.i.d. Gaussian     

Quantization based on a novel sample-adaptive product quantizer(SAPQ)

In this paper, we propose a novel feedforward adaptive quantization scheme called the sample-adaptive product quantizer (SAPQ). This is a structurally constrained vector quantizer that uses unions of product codebooks. SAPQ is based on a concept of adaptive quantization to the varying samples of the source and is very different from traditional adaptation techniques for nonstationary sources. SAPQ quantizes each source sample using a sequence of quantizers. Even when using scalar quantization in SAPQ, we can achieve performance comparable to vector quantization (with the complexity still close to that of scalar quantization). We also show that important lattice-based vector quantizers can be constructed using scalar quantization in SAPQ. We mathematically analyze SAPQ and propose a algorithm to implement it. We numerically study SAPQ for independent and identically distributed Gaussian and Laplacian sources. Through our numerical study, we find that SAPQ using scalar quantizers achieves typical gains of 13 dB in distortion over the Lloyd-Max quantizer. We also show that SAPQ can he used in conjunction with vector quantizers to further improve the gains     

Regularized subband coding scheme

Two enhanced subband coding schemes using a regularized image restoration technique are proposed: the first controls the global regularity of the decompressed image; the second extends the first approach at each decomposition level. The quantization scheme incorporates scalar quantization (SQ) and pyramidal lattice vector quantization (VQ) with both optimal bit and quantizer allocation. Experimental results show that both the block effect due to VQ and the quantization noise are significantly reduced.     

Sample-adaptive product quantization: asymptotic analysis andexamples

Vector quantization (VQ) is an efficient data compression technique for low bit rate applications. However the major disadvantage of VQ is that its encoding complexity increases dramatically with bit rate and vector dimension. Even though one can use a modified VQ, such as the tree-structured VQ, to reduce the encoding complexity, it is practically infeasible to implement such a VQ at a high bit rate or for large vector dimensions because of the huge memory requirement for its codebook and for the very large training sequence requirement. To overcome this difficulty, a structurally constrained VQ called the sample-adaptive product quantizer (SAPQ) has recently been proposed. We extensively study the SAPQ that is based on scalar quantizers in order to exploit the simplicity of scalar quantization. Through an asymptotic distortion result, we discuss the achievable performance and the relationship between distortion and encoding complexity. We illustrate that even when SAPQ is based on scalar quantizers, it can provide VQ-level performance. We also provide numerical results that show a 2-3 dB improvement over the Lloyd-Max (1982, 1960) quantizers for data rates above 4 b/point     

Sequential scalar quantization of vectors: an analysis

Proposes an efficient vector quantization (VQ) technique called sequential scalar quantization (SSQ). The scalar components of the vector are individually quantized in a sequence, with the quantization of each component utilizing conditional information from the quantization of previous components. Unlike conventional independent scalar quantization (ISQ), SSQ has the ability to exploit intercomponent correlation. At the same time, since quantization is performed on scalar rather than vector variables, SSQ offers a significant computational advantage over conventional VQ techniques and is easily amenable to a hardware implementation. In order to analyze the performance of SSQ, the authors appeal to asymptotic quantization theory, where the codebook size is assumed to be large. Closed-form expressions are derived for the quantizer mean squared error (MSE). These expressions are used to compare the asymptotic performance of SSQ with other VQ techniques. The authors also demonstrate the use of asymptotic theory in designing SSQ for a practical application (color image quantization), where the codebook size is typically small. Theoretical and experimental results show that SSQ far outperforms ISQ with respect to MSE while offering a considerable reduction in computation over conventional VQ at the expense of a moderate increase in MSE.     

A fuzzy classified vector quantizer for image coding

Vector quantization of images raises problems of complexity in codebook search and subjective quality of images. The family of image vector quantization algorithms proposed in this paper addresses both of those problems. The fuzzy classified vector quantizer (FCVQ) is based on fuzzy set theory and consists basically in a method of extracting a subcodebook from the original codebook, biased by the features of the block to be coded. The incidence of each feature on the blocks is represented by a fuzzy set that captures its (possibly subjective) nature. Unlike the classified vector quantizer (CVQ), in the FCVQ a specific subcodebook is extracted for each block to be coded, allowing a better adaptation to the block. The CVQ may be regarded as a special case of the FCVQ. In order to explore the possible correlation between blocks, an estimator for the degree of incidence of features on the block to be coded is included. The estimate is based on previously coded blocks and is obtained by maximizing a possibility; a distribution that intends to represent the subjective knowledge on the feature's possibility of occurrence conditioned to the coded blocks is used. Some examples of the application of a FCVQ coder to two test images are presented. A slight improvement on the subjective quality of the coded images is obtained, together with a significant reduction on the codebook search complexity and, when applying the estimator, a reduction of the bit rate     

Robust vector quantization by a linear mapping of a block code

In this paper we propose a novel technique for vector quantizer design where the reconstruction vectors are given by a linear mapping of a binary block code (LMBC). The LMBC framework provides a relation between the index bits and the reconstruction vectors through mapping properties. We define a framework, show its flexibility, and give optimality conditions. We consider source optimized vector quantization (VQ), where the objective is to directly obtain a VQ with inherent good channel robustness properties. Several instructive theoretical results and properties of the distortion experienced due to channel noise are demonstrated. These results are used to guide the design process. Both optimization algorithms and a block code selection procedure are devised. Experimental results for Gauss-Markov sources show that quantization performance close to an unconstrained VQ is obtained with a short block code which implies a constrained VQ. The resulting VQs have better channel noise robustness than conventional VQs designed with the generalized Lloyd algorithm (GLA) and splitting initialization, even when a post-processing index assignment algorithm is applied to the GLA-based VQ. We have, thus, demonstrated a unique method for direct design resulting in an inherent good index assignment combined with small losses in quantization performance     

Design of sample adaptive product quantizers for noisy channels

Channel-optimized vector quantization (COVQ) has proven to be an effective joint source-channel coding technique that makes the underlying quantizer robust to channel noise. Unfortunately, COVQ retains the high encoding complexity of the standard vector quantizer (VQ) for medium-to-high quantization dimensions and moderate-to-good channel conditions. A technique called sample adaptive product quantization (SAPQ) was recently introduced by Kim and Shroff to reduce the complexity of the VQ while achieving comparable distortions. In this letter, we generalize the design of SAPQ for the case of memoryless noisy channels by optimizing the quantizer with respect to both source and channel statistics. Numerical results demonstrate that the channel-optimized SAPQ (COSAPQ) achieves comparable performance to the COVQ (within 0.2 dB), while maintaining considerably lower encoding complexity (up to half of that of COVQ) and storage requirements. Robustness of the COSAPQ system against channel mismatch is also examined.     

Asymptotic performance of vector quantizers with a perceptualdistortion measure

Gersho's (1979) bounds on the asymptotic performance of vector quantizers are valid for vector distortions which are powers of the Euclidean norm. Yamada, Tazaki, and Gray (1980) generalized the results to distortion measures that are increasing functions of the norm of their argument. In both cases, the distortion is uniquely determined by the vector quantization error, i.e., the Euclidean difference between the original vector and the codeword into which it is quantized. We generalize these asymptotic bounds to input-weighted quadratic distortion measures and measures that are approximately output-weighted-quadratic when the distortion is small, a class of distortion measures often claimed to be perceptually meaningful. An approximation of the asymptotic distortion based on Gersho's conjecture is derived as well. We also consider the problem of source mismatch, where the quantizer is designed using a probability density different from the true source density. The resulting asymptotic performance in terms of distortion increase in decibels is shown to be linear in the relative entropy between the true and estimated probability densities     

Gaussian source coding with spherical codes

A fixed-rate shape-gain quantizer for the memoryless Gaussian source is proposed. The shape quantizer is constructed from wrapped spherical codes that map a sphere packing in ℝ^k-1 onto a sphere in ℝ^k, and the gain codebook is a globally optimal scalar quantizer. A wrapped Leech lattice shape quantizer is used to demonstrate a signal-to-quantization-noise ratio within 1 dB of the distortion-rate function for rates above 1 bit per sample, and an improvement over existing techniques of similar complexity. An asymptotic analysis of the tradeoff between gain quantization and shape quantization is also given     

A vector quantizer for the Laplace source

The low complexity, nearly optimal vector quantizer (VQ) is a generalization of T. R. Fischer's (1986) pyramid VQ and is similar in structure to the unrestricted polar quantizers previously presented for the independent Gaussian source. An analysis of performance is presented with results for both the product code pyramid VQ and the unrestricted version. This analysis, although asymptotic in nature, helps to demonstrate the performance advantages of the VQ. Implementation issues of the VQ are discussed. Nonasymptotic results are considered. In particular, the author presents an approximate design algorithm for finite bit rate and demonstrates the usefulness of this VQ through several example designs with Monte Carlo simulations of performance. For the restricted form (the pyramid VQ), the author provides further implementational information and low dimension analytical results     

High-resolution quantization theory and the vector quantizeradvantage

The authors consider how much performance advantage a fixed-dimensional vector quantizer can gain over a scalar quantizer. They collect several results from high-resolution or asymptotic (in rate) quantization theory and use them to identify source and system characteristics that contribute to the vector quantizer advantage. One well-known advantage is due to improvement in the space-filling properties of polytopes as the dimension increases. Others depend on the source's memory and marginal density shape. The advantages are used to gain insight into product, transform, lattice, predictive, pyramid, and universal quantizers. Although numerical prediction consistently overestimated gains in low rate (1 bit/sample) experiments, the theoretical insights may be useful even at these rates     

A complexity reduction technique for image vector quantization

A technique for reducing the complexity of spatial-domain image vector quantization (VQ) is proposed. The conventional spatial domain distortion measure is replaced by a transform domain subspace distortion measure. Due to the energy compaction properties of image transforms, the dimensionality of the subspace distortion measure can be reduced drastically without significantly affecting the performance of the new quantizer. A modified LBG algorithm incorporating the new distortion measure is proposed. Unlike conventional transform domain VQ, the codevector dimension is not reduced and a better image quality is guaranteed. The performance and design considerations of a real-time image encoder using the techniques are investigated. Compared with spatial domain a speed up in both codebook design time and search time is obtained for mean residual VQ, and the size of fast RAM is reduced by a factor of four. Degradation of image quality is less than 0.4 dB in PSNR.     

Asymptotic bounds on optimal noisy channel quantization via randomcoding

Asymptotically optimal zero-delay vector quantization in the presence of channel noise is studied using random coding techniques. First, an upper bound is derived for the average rth-power distortion of channel optimized k-dimensional vector quantization at transmission rate R on a binary symmetric channel with bit error probability ϵ. The upper bound asymptotically equals 2/sup -rRg(ϵ,k,r/). where k/(k +r) [1 - log₂(l +2√(ϵ(1-ϵ))] ⩽g(ϵ,k,r)⩽1) for all ϵ⩾0, lim_ϵ→0 g(ϵ,k,r)=1, and lim_k→∞g(ϵ,k,r)=1. Numerical computations of g(ϵ,k,r) are also given. This result is analogous to Zador's (1982) asymptotic distortion rate of 2^-rR for quantization on noiseless channels. Next, using a random coding argument on nonredundant index assignments, a useful upper bound is derived in terms of point density functions, on the minimum mean squared error of high resolution, regular, vector quantizers in the presence of channel noise. The formula provides an accurate approximation to the distortion of a noisy channel quantizer whose codebook is arbitrarily ordered. Finally, it is shown that the minimum mean squared distortion of a regular, noisy channel VQ with a randomized nonredundant index assignment, is, in probability, asymptotically bounded away from zero