Variance mismatch of vector quantizers (Corresp.)

The asymptotic performance of variance-mismatched vector quantizers is presented. It is demonstrated by both asymptotic analysis and computer simulations that well-designed vector quantizers are inherently more invulnerable to mismatch than are conventional scalar quantizers. A generalized exponential density function is considered as a statistical model of sources. As an example, the asymptotic performance is derived and applied to the memoryless Laplacian source with the squared-error distortion measure.     

On the structure of optimal entropy-constrained scalar quantizers

The nearest neighbor condition implies that when searching for a mean-square optimal fixed-rate quantizer it is enough to consider the class of regular quantizers, i.e., quantizers having convex cells and codepoints which lie inside the associated cells. In contrast, quantizer regularity can preclude optimality in entropy-constrained quantization. This can be seen by exhibiting a simple discrete scalar source for which the mean-square optimal entropy-constrained scalar quantizer (ECSQ) has disconnected (and hence nonconvex) cells at certain rates. In this work, new results concerning the structure and existence of optimal ECSQs are presented. One main result shows that for continuous sources and distortion measures of the form d(x,y)=ρ(|x-y|), where ρ is a nondecreasing convex function, any finite-level ECSQ can be "regularized" so that the resulting regular quantizer has the same entropy and equal or less distortion. Regarding the existence of optimal ECSQs, we prove that under rather general conditions there exists an "almost regular" optimal ECSQ for any entropy constraint. For the squared error distortion measure and sources with piecewise-monotone and continuous densities, the existence of a regular optimal ECSQ is shown     

Codecell convexity in optimal entropy-constrained vector quantization

Properties of optimal entropy-constrained vector quantizers (ECVQs) are studied for the squared-error distortion measure. It is known that restricting an ECVQ to have convex codecells may preclude its optimality for some sources with discrete distribution. We show that for sources with continuous distribution, any finite-level ECVQ can be replaced by another finite-level ECVQ with convex codecells that has equal or better performance. We generalize this result to infinite-level quantizers, and also consider the problem of existence of optimal ECVQs for continuous source distributions. In particular, we show that given any entropy constraint, there exists an ECVQ with (possibly infinitely many) convex codecells that has minimum distortion among all ECVQs satisfying the constraint. These results extend analogous statements in entropy-constrained scalar quantization. They also generalize results in entropy-constrained vector quantization that were obtained via the Lagrangian formulation and, therefore, are valid only for certain values of the entropy constraint.     

Finite-state vector quantization for waveform coding

A finite-state vector quantizer is a finite-state machine used for data compression: Each successive source vector is encoded into a codeword using a minimum distortion rule, and into a code book, depending on the encoder state. The current state and the selected codeword then determine the next encoder state. A finite-state vector quantizer is capable of making better use of the memory in a source than is an ordinary memoryless vector quantizer of the same dimension or blocklength. Design techniques are introduced for finite-state vector quantizers that combine ad hoc algorithms with an algorithm for the design of memoryless vector quantizers. Finite-state vector quantizers are designed and simulated for Gauss-Markov sources and sampled speech data, and the resulting performance and storage requirements are compared with ordinary memoryless vector quantization.     

On the performance and complexity of channel-optimized vectorquantizers

The performance and complexity of channel-optimized vector quantizers are studied for the Gauss-Markov source. Observations on the geometric structure of these quantizers are made, which have an important implication on the encoding complexity. For the squared-error distortion measure, it is shown that an operation equivalent to a Euclidean distance measurement with respect to an appropriately defined set of points (used to identify the encoding regions) can be used to perform the encoding. This implies that the encoding complexity is proportional to the number of encoding regions. It is then demonstrated that for very noisy channels and a heavily correlated source, when the codebook size is large, the number of encoding regions is considerably smaller than the codebook size-implying a reduction in encoding complexity     

An Algorithm for the Design of Labeled-Transition Finite-State Vector Quantizers

A finite-state vector quantizer (FSVQ) is a switched vector quantizer where the sequence of quantizers selected by the encoder can be tracked by the decoder. It can be viewed as an adaptive vector quantizer with backward estimation, a vector generalization of an AQB system. Recently a family of algorithms for the design of FSVQ's for waveform coding application has been introduced. These algorithms first design an initial set of vector quantizers together with a next-state function giving the rule by which the next quantizer is selected. The codebooks of this initial FSVQ are then iteratively improved by a natural extension of the usual memoryless vector quantizer design algorithm. The next-state function, however, is not modified from its initial form. In this paper we present two extensions of the FSVQ design algorithms. First, the algorithm for FSVQ design for waveform coders is extended to FSVQ design of linear predictive coded (LPC) speech parameter vectors using an Itakura-Saito distortion measure. Second, we introduce a new technique for the iterative improvement of the next-state function based on an algorithm from adaptive stochastic automata theory. The design algorithms are simulated for an LPC FSVQ and the results are compared with each other and to ordinary memoryless vector quantization. Several open problems suggested by the simulation results are presented.     

An algorithm for uniform vector quantizer design

A vector quantizer maps ak-dimensional vector into one of a finite set of output vectors or "points". Although certain lattices have been shown to have desirable properties for vector quantization applications, there are as yet no algorithms available in the quantization literature for building quantizers based on these lattices. An algorithm for designing vector quantizers based on the root latticesA_{n}, D_{n}, andE_{n}and their duals is presented. Also, a coding scheme that has general applicability to all vector quantizers is presented. A four-dimensional uniform vector quantizer is used to encode Laplacian and gamma-distributed sources at entropy rates of one and two bits/sample and is demonstrated to achieve performance that compares favorably with the rate distortion bound and other scalar and vector quantizers. Finally, an application using uniform four- and eight-dimensional vector quantizers for encoding the discrete cosine transform coefficients of an image at0.5bit/pel is presented, which visibly illustrates the performance advantage of vector quantization over scalar quantization.     

Geometric source coding and vector quantization

A geometric formulation is presented for source coding and vector quantizer design. Motivated by the asymptotic equipartition principle, the authors consider two broad classes of source codes and vector quantizers: elliptical codes and quantizers based on the Gaussian density function, and pyramid codes and quantizers based on the Laplacian density function. Elliptical and weighted pyramid vector quantizers are developed by selecting codewords as points in a lattice that lie on (or near) a specified ellipse or pyramid. The combination of geometric structure and lattice basis allows simple encoding and decoding algorithms     

Quantization based on a novel sample-adaptive product quantizer(SAPQ)

In this paper, we propose a novel feedforward adaptive quantization scheme called the sample-adaptive product quantizer (SAPQ). This is a structurally constrained vector quantizer that uses unions of product codebooks. SAPQ is based on a concept of adaptive quantization to the varying samples of the source and is very different from traditional adaptation techniques for nonstationary sources. SAPQ quantizes each source sample using a sequence of quantizers. Even when using scalar quantization in SAPQ, we can achieve performance comparable to vector quantization (with the complexity still close to that of scalar quantization). We also show that important lattice-based vector quantizers can be constructed using scalar quantization in SAPQ. We mathematically analyze SAPQ and propose a algorithm to implement it. We numerically study SAPQ for independent and identically distributed Gaussian and Laplacian sources. Through our numerical study, we find that SAPQ using scalar quantizers achieves typical gains of 13 dB in distortion over the Lloyd-Max quantizer. We also show that SAPQ can he used in conjunction with vector quantizers to further improve the gains     

Bennett's integral for vector quantizers

This paper extends Bennett's (1948) integral from scalar to vector quantizers, giving a simple formula that expresses the rth-power distortion of a many-point vector quantizer in terms of the number of points, point density function, inertial profile, and the distribution of the source. The inertial profile specifies the normalized moment of inertia of quantization cells as a function of location. The extension is formulated in terms of a sequence of quantizers whose point density and inertial profile approach known functions as the number of points increase. Precise conditions are given for the convergence of distortion (suitably normalized) to Bennett's integral. Previous extensions did not include the inertial profile and, consequently, provided only bounds or applied only to quantizers with congruent cells, such as lattice and optimal quantizers. The new version of Bennett's integral provides a framework for the analysis of suboptimal structured vector quantizers. It is shown how the loss in performance of such quantizers, relative to optimal unstructured ones, can be decomposed into point density and cell shape losses. As examples, these losses are computed for product quantizers and used to gain further understanding of the performance of scalar quantizers applied to stationary, memoryless sources and of transform codes applied to Gaussian sources with memory. It is shown that the short-coming of such quantizers is that they must compromise between point density and cell shapes     

A vector quantization approach to universal noiseless coding andquantization

A two-stage code is a block code in which each block of data is coded in two stages: the first stage codes the identity of a block code among a collection of codes, and the second stage codes the data using the identified code. The collection of codes may be noiseless codes, fixed-rate quantizers, or variable-rate quantizers. We take a vector quantization approach to two-stage coding, in which the first stage code can be regarded as a vector quantizer that “quantizes” the input data of length n to one of a fixed collection of block codes. We apply the generalized Lloyd algorithm to the first-stage quantizer, using induced measures of rate and distortion, to design locally optimal two-stage codes. On a source of medical images, two-stage variable-rate vector quantizers designed in this way outperform standard (one-stage) fixed-rate vector quantizers by over 9 dB. The tail of the operational distortion-rate function of the first-stage quantizer determines the optimal rate of convergence of the redundancy of a universal sequence of two-stage codes. We show that there exist two-stage universal noiseless codes, fixed-rate quantizers, and variable-rate quantizers whose per-letter rate and distortion redundancies converge to zero as (k/2)n ^-1 log n, when the universe of sources has finite dimension k. This extends the achievability part of Rissanen's theorem from universal noiseless codes to universal quantizers. Further, we show that the redundancies converge as O(n^-1) when the universe of sources is countable, and as O(n^-1+ϵ) when the universe of sources is infinite-dimensional, under appropriate conditions     

Recursively indexed quantization of memoryless sources

A recursively indexed scalar quantizer that performs as well as high-dimensional vector quantizers for several important sources, without the attendant complexity, is presented. The recursively indexed quantizer provides a simple technique, both in terms of design and operation, for use with entropy coding. It also provides a simple quantization technique for use in noisy channel conditions where a variable length code would be inappropriate but performance greater than that provided by Lloyd-Max quantizers is desired     

An algorithm for the design of generalized quantizers for detection

A procedure based on the generalized Lloyd algorithm approach using a sequence of independent noise samples to design M-region generalized quantizers for signal detection is presented. Included in this case are the conventional M-interval quantizer detectors. The quantizer parameters for S.A. Kassam's (1985) four-region generalized quantizer detector are computed using various sample sizes for the known sequence of independent noise samples. Two families of densities which cover a wide spectrum of possible nonGaussian densities are considered: the generalized Gaussian densities and the Johnson S_u family of densities. The performance of the quantizer detector is compared to that of the locally optimum detector, and the results are presented as the asymptotic relative efficiencies of the respective detectors. The case when the noise density is not known before analysis is considered, and the detection performance is examined using an estimate of the density. A mean-squared-error distortion criterion is used in the proposed algorithm to obtain quantizers that yield maximum efficacy. It is shown through numerical examples that the design procedure is simple, fast, and applicable to a wide range of nonGaussian distributions     

Transmission of analog signals using multicarrier modulation: acombined source-channel coding approach

We have proposed and analyzed a combined source-channel coding scheme using multicarrier modulation. By changing the power and modulation of subchannels carrying different bits of the quantized signal, the channel-induced distortion can be minimized. We derive an algorithm for the allocation of power among subchannels. Analog sources quantized by scalar and vector quantizers are considered. For a Gaussian source scalar-quantized to 8 bit, multicarrier transmission yields a 13 dB improvement of signal-to-distortion ratio over a single-carrier system. Quantizers having a smaller number of bits yield smaller improvements. We consider both full-search and binary tree-search vector quantizers using a natural binary coding scheme. For a vector quantizer using 2 b/sample, multicarrier transmission yields an improvement in signal-to-distortion ratio that lies between 0.3 and 0.8 dB, depending on the quantizer design     

A unified approach for encoding clean and noisy sources by means ofwaveform and autoregressive model vector quantization

Data compression by vector quantization is considered for sources which have been degraded by noise. It is shown that, by appropriately modifying the given distortion measure, the problem becomes a standard quantization problem for the noisy source and the modified distortion measure. For the special case of sources corrupted by statistically independent additive noise, the authors provide sufficient conditions on the original distortion measure and probability distributions of the source and the noise for convergence of the generalized Lloyd algorithm in designing the quantizers. The results are specialized to waveform and autoregressive model vector quantization using the weighted quadratic and the Itakura-Saito distortion measures, respectively     

On finite-state vector quantization for noisy channels

Finite-state vector quantization (FSVQ) over a noisy channel is studied. A major drawback of a finite-state decoder is its inability to track the encoder in the presence of channel noise. In order to overcome this problem, we propose a nontracking decoder which directly estimates the code vectors used by a finite-state encoder. The design of channel-matched finite-state vector quantizers for noisy channels, using an iterative scheme resembling the generalized Lloyd algorithm, is also investigated. Simulation results based on encoding a Gauss-Markov source over a memoryless Gaussian channel show that the proposed decoder exhibits graceful degradation of performance with increasing channel noise, as compared with a finite-state decoder. Also, the channel-matched finite-state vector quantizers are shown to outperform channel-optimized vector quantizers having the same vector dimension and rate. However, the nontracking decoder used in the channel-matched finite-state quantizer has a higher computational complexity, compared with a channel-optimized vector-quantizer decoder. Thus, if they are allowed to have the same overall complexity (encoding and decoding), the channel-optimized vector quantizer can use a longer encoding delay and achieve similar or better performance. Finally, an example of using the channel-matched finite-state quantizer as a backward-adaptive quantizer for nonstationary signals is also presented.     

The minimax distortion redundancy in empirical quantizer design

We obtain minimax lower and upper bounds for the expected distortion redundancy of empirically designed vector quantizers. We show that the mean-squared distortion of a vector quantizer designed from n independent and identically distributed (i.i.d.) data points using any design algorithm is at least Ω(n^-1/2) away from the optimal distortion for some distribution on a bounded subset of ℛ ^d. Together with existing upper bounds this result shows that the minimax distortion redundancy for empirical quantizer design, as a function of the size of the training data, is asymptotically on the order of n^-1/2. We also derive a new upper bound for the performance of the empirically optimal quantizer     

A pyramid vector quantizer

The geometric properties of a memoryless Laplacian source are presented and used to establish a source coding theorem. Motivated by this geometric structure, a pyramid vector quantizer (PVQ) is developed for arbitrary vector dimension. The PVQ is based on the cubic lattice points that lie on the surface of anL-dimensional pyramid and has simple encoding and decoding algorithms. A product code version of the PVQ is developed and generalized to apply to a variety of sources. Analytical expressions are derived for the PVQ mean square error (mse), and simulation results are presented for PVQ encoding of several memoryless sources. For large rate and dimension, PVQ encoding of memoryless Laplacian, gamma, and Gaussian sources provides rose improvements of5.64, 8.40, and2.39dB, respectively, over the corresponding optimum scalar quantizer. Although suboptimum in a rate-distortion sense, because the PVQ can encode large-dimensional vectors, it offers significant reduction in rose distortion compared with the optimum Lloyd-Max scalar quantizer, and provides an attractive alternative to currently available vector quantizers.