Improving the performance of k-means for color quantization

Color quantization is an important operation with many applications in graphics and image processing. Most quantization methods are essentially based on data clustering algorithms. However, despite its popularity as a general purpose clustering algorithm, k-means has not received much respect in the color quantization literature because of its high computational requirements and sensitivity to initialization. In this paper, we investigate the performance of k-means as a color quantizer. We implement fast and exact variants of k-means with several initialization schemes and then compare the resulting quantizers to some of the most popular quantizers in the literature. Experiments on a diverse set of images demonstrate that an efficient implementation of k-means with an appropriate initialization strategy can in fact serve as a very effective color quantizer.     

Bit plane modification for improving MSE-near optimal DPCM-based block truncation coding

A hybrid method of block truncation coding (BTC) and differential pulse code modulation (DPCM) offers better visual quality than the standard BTC for small block sizes due to its inherent multitone representation. Recently, a two-level quantizer design method has been proposed to increase the coding performance of the DPCM-BTC framework. However, the design method is near optimal in the sense that its coding performance depends on the initial bit plane patterns. In this paper, we propose a bit plane modification (BPM) algorithm to achieve further performance improvement. The BPM algorithm, inspired by error diffusion, effectively distributes large quantization error at a certain pixel to its neighboring pixels having small quantization errors by changing partial bit patterns. Experimental results show that the proposed algorithm successfully achieves much higher coding performance than various conventional BTC methods. The average PSNR performance of the proposed method is 2.31 dB, 5.15 dB, and 5.15 dB higher than that of BTC, DPCM-BTC, and a recently developed BTC scheme using error diffusion and bilateral filtering, respectively.     

Optimal entropy-constrained non-uniform scalar quantizer design for low bit-rate pixel domain DVC

In this paper, an optimal entropy-constrained non-uniform scalar quantizer is proposed for the pixel domain DVC. The uniform quantizer is efficient for the hybrid video coding since the residual signals conforming to a single-variance Laplacian distribution. However, the uniform quantizer is not optimal for pixel domain distributed video coding (DVC). This is because the uniform quantizer is not adaptive to the joint distribution of the source and the SI, especially for low level quantization. The signal distribution of pixel domain DVC conforms to the mixture model with multi-variance. The optimal non-uniform quantizer is designed according to the joint distribution, the error between the source and the SI can be decreased. As a result, the bit rate can be saved and the video quality won’t sacrifice too much. Accordingly, a better R-D trade-off can be achieved. First, the quantization level is fixed and the optimal RD trade-off is achieved by using a Lagrangian function J(Q). The rate and distortion components is designed based on P(Y|Q). The conditional probability density function of SI Y depend on quantization partitions Q, P(Y|Q), is approximated by a Guassian mixture model at encocder. Since the SI can not be accessed at encoder, an estimation of P(Y|Q) based on the distribution of the source is proposed. Next, J(Q) is optimized by an iterative Lloyd-Max algorithm with a novel quantization partition updating algorithm. To guarantee the convergence of J(Q), the monotonicity of the interval in which the endpoints of the quantizer lie must be satisfied. Then, a quantizer partition updating algorithm which considers the extreme points of the histogram of the source is proposed. Consequently, the entropy-constrained optimal non-uniform quantization partitions are derived and a better RD trade-off is achieved by applying them. Experiment results show that the proposed scheme can improve the performance by 0.5 dB averagely compared to the uniform scalar quantization.     

Combining image compression and classification using vectorquantization

We describe a method of combining classification and compression into a single vector quantizer by incorporating a Bayes risk term into the distortion measure used in the quantizer design algorithm. Once trained, the quantizer can operate to minimize the Bayes risk weighted distortion measure if there is a model providing the required posterior probabilities, or it can operate in a suboptimal fashion by minimizing the squared error only. Comparisons are made with other vector quantizer based classifiers, including the independent design of quantization and minimum Bayes risk classification and Kohonen's LVQ. A variety of examples demonstrate that the proposed method can provide classification ability close to or superior to learning VQ while simultaneously providing superior compression performance     

Geometric piecewise uniform lattice vector quantization of the memoryless Gaussian source

The aim of this paper is to find a quantization technique that has low implementation complexity and asymptotic performance arbitrarily close to the optimum. More specifically, it is of interest to develop a new vector quantizer design procedure for a memoryless Gaussian source that yields vector quantizers with excellent performance and the structure required for fast quantization. To achieve this, we combined a fast lattice-encoding algorithm with a geometric approach to generate a model of a geometric piecewise-uniform lattice vector quantizer. Expressions for granular distortion and the optimal number of outputs points in each region were derived. Both exact and approximative asymptotic analyses were carried out. During this process, the constant probability density function of the input signal vector was kept inside the whole region. The analysis demonstrated the existence of piecewise-constant approximations to the input-vector probability density function, which is optimal for the proposed geometric piecewise-uniform vector quantizer. The considered quantization technique is near optimal for a memoryless Gaussian source. In other words, this paper proposes a method for a near-optimum, low-complex vector quantizer design based on probability density function discretization. The presented methodology gives a signal-to-quantization noise ratio that in some cases differs from the optimum by 0.1 dB or less. Improvements of the considered model in performance and complexity over some of the existing techniques are also demonstrated.     

A VLSI neural processor for image data compression usingself-organization networks

An adaptive electronic neural network processor has been developed for high-speed image compression based on a frequency-sensitive self-organization algorithm. The performance of this self-organization network and that of a conventional algorithm for vector quantization are compared. The proposed method is quite efficient and can achieve near-optimal results. The neural network processor includes a pipelined codebook generator and a paralleled vector quantizer, which obtains a time complexity O(1) for each quantization vector. A mixed-signal design technique with analog circuitry to perform neural computation and digital circuitry to process multiple-bit address information are used. A prototype chip for a 25-D adaptive vector quantizer of 64 code words was designed, fabricated, and tested. It occupies a silicon area of 4.6 mmx6.8 mm in a 2.0 mum scalable CMOS technology and provides a computing capability as high as 3.2 billion connections/s. The experimental results for the chip and the winner-take-all circuit test structure are presented.     

An efficient iterative algorithm for designing an asymptotically optimal modified unrestricted uniform polar quantization of bivariate Gaussian random variables

In this paper, a more efficient and a more accurate algorithm is developed for designing asymptotically optimal unrestricted uniform polar quantization (UUPQ) of bivariate Gaussian random variables compared to the existing algorithms on this subject. The proposed algorithm is an iterative one defining the analytical model of asymptotically optimal UUPQ in only a few iterations. The UUPQ model is also improved via optimization of the last magnitude reconstruction level so that the mean squared error (MSE) is minimal. Moreover, for the straightforward performance assessment of our analytical UUPQ model an asymptotic formula for signal to quantization noise ratio (SQNR) is derived, which is reasonably accurate for any rate (R) greater than or equal to 2.5 bits/sample. It is demonstrated empirically that our asymptotically optimal UUPQ model outperforms the previous UUPQ models in terms of SQNR. Eventually, the transition from the analytical to the practically designed UUPQ model, as an important aspect in quantizer design, is considered in the paper and, as a result, a novel method to achieve this is provided. The proposed method is applicable to the practical design of any unrestricted polar quantization.     

State estimation for linear discrete-time systems using quantized measurements

In this paper, we consider the problem of state estimation for linear discrete-time dynamic systems using quantized measurements. This problem arises when state estimation needs to be done using information transmitted over a digital communication channel. We investigate how to design the quantizer and the estimator jointly. We consider the use of a logarithmic quantizer, which is motivated by the fact that the resulting quantization error acts as a multiplicative noise, an important feature in many applications. Both static and dynamic quantization schemes are studied. The results in the paper allow us to understand the tradeoff between performance degradation due to quantization and quantization density (in the infinite-level quantization case) or number of quantization levels (in the finite-level quantization case).     

Image retrieval using color histograms generated by Gauss mixture vector quantization

Image retrieval based on color histograms requires quantization of a color space. Uniform scalar quantization of each color channel is a popular method for the reduction of histogram dimensionality. With this method, however, no spatial information among pixels is considered in constructing the histograms. Vector quantization (VQ) provides a simple and effective means for exploiting spatial information by clustering groups of pixels. We propose the use of Gauss mixture vector quantization (GMVQ) as a quantization method for color histogram generation. GMVQ is known to be robust for quantizer mismatch, which motivates its use in making color histograms for both the query image and the images in the database. Results show that the histograms made by GMVQ with a penalized log-likelihood (LL) distortion yield better retrieval performance for color images than the conventional methods of uniform quantization and VQ with squared error distortion.     

Optimization of multiple region quantizer for Laplacian source

This paper proposes a multiple region quantizer composed of quantizers defined on different disjunctive regions of an input signal. In particular, for the two region and the three region cases, the paper provides a complete optimization of a multiple region companded quantizer for the Laplacian source of unit variance. The analysis of the multiple region quantizer is limited to a three region case due to the complexity of the optimization problem and due to the fact that much more complex multiple region quantizer models obtained for a higher number of regions could slightly improve the performances. Two-stage optimization is performed with respect to the number of reconstruction levels of each quantizer composing the considered multiple region companded quantizer and with respect to the region bounds. It is shown that optimal parameters depend only on the fractional part of the required average bit rate. In order to design the three region optimal quantizer, Lloyd–Max's algorithm and Newton–Kantorovich iterative method are used with the three region optimal companded quantizer as the initial solution. The gradient Newton–Kantorovich iterative method is used to provide better convergence speed than Lloyd–Max's algorithm, which is essential in cases where effective initialization solution of Lloyd–Max's algorithm is missing. It is shown that the three region optimal companded quantizer have signal to quantization noise ratio value close to the one of the three region optimal quantizer, where a simpler design procedure is the benefit of the three region optimal companded quantizer over the three region optimal one.     

随机性参数分布式量化估计及其最优比特分配

本文研究总比特率给定下随机向量参数分布式量化估计及其最优比特分配问题.与现有文献大都假定每个传感器的量化比特率给定而不是最优分配下研究随机性参数的分布式量化估计问题不同的是,本文将综合考虑最优量化器、最优估计器算法以及给定总比特率下的最优比特分配问题.针对向量状态标量观测模型,首先借助现有文献给出基于量化观测的最优估计器及其误差协方差阵形式表达,其次得到各传感器的渐近最优量化器实际为著名的Lloyd-max量化器,且各传感器的渐近最优量化级数与信噪比成正比,同时引入一种次优的求解非负整数比特率的方法.考虑到当传感器数目比较大时,初始的最优估计器算法运算量很大,设计了一种渐近等价的迭代量化估计器算法,其计算负担大大减轻,且对于存在延迟或丢包的网络环境亦适用,增强了算法的鲁棒性.仿真结果表明,本文提出的最优比特分配方案估计性能明显优于一般的均匀比特分配方案.     

Fuzzy vector quantization for image compression based on competitive agglomeration and a novel codeword migration strategy

The implementation of fuzzy clustering in the design process of vector quantizers faces three challenges. The first is the high computational cost. The second challenge arises because a vector quantizer is required to assign each training sample to only one cluster. However, such an aggressive interpretation of fuzzy clustering results to a crisp partition of inferior quality. The third one is the dependence on initialization. In this paper we develop a fuzzy clustering-based vector quantization algorithm that deals with the aforementioned problems. The algorithm utilizes a specialized objective function, which involves the c-means and the fuzzy c-means along with a competitive agglomeration term. The joint effect is a learning process where the number of codewords (i.e. cluster centers) affected by a specific training sample is gradually reducing and therefore, the number of distance calculations is also reducing. Thus, the computational cost becomes smaller. In addition, the partition is smoothly transferred from fuzzy to crisp conditions and there is no need to employ any aggressive interpretation of fuzzy clustering. The competitive agglomeration term refines large clusters from small and spurious ones. Then, contrary to the classical competitive agglomeration method, we do not discard the small clusters but instead migrate them close to large clusters, rendering more competitive. Thus, the codeword migration process uses the net effect of the competitive agglomeration and acts to further reduce the dependence on initialization in order to obtain a better local minimum. The algorithm is applied to grayscale image compression. The main simulation findings can be summarized as follows: (a) a comparison between the proposed method and other related approaches shows its statistically significant superiority, (b) the algorithm is a fast process, (c) the algorithm is insensitive with respect to its design parameters, and (d) the reconstructed images maintain high quality, which is quantified in terms of the distortion measure.     

A fast VQ codebook search with initialization and search order

Vector quantization (VQ), a lossy image compression, is widely used for many applications due to its simple architecture, fast decoding ability, and high compression rate. Traditionally, VQ applies the full search algorithm to search for the codeword that best matches each image vector in the encoding procedure. However, matching in this manner consumes a lot of computation time and leads to a heavy burden for the VQ method. Therefore, Torres and Huguet proposed a double test algorithm to improve the matching efficiency. However, their scheme does not include an initiation strategy to choose an initially searched codeword for each image vector, and, as a result, matching efficiency may be affected significantly. To overcome this drawback, we propose an improved double test scheme with a fine initialization as well as a suitable search order. Our experimental results indicate that the computation time of the double test algorithm can be significantly reduced by the proposed method. In addition, the proposed method is more flexible than existing schemes.     

Energy-Based Split Vector Quantizer Employing Signal Representation in Multiple Transform Domains

Mikhael, W., and Krishnan, V., Energy-Based Split Vector Quantizer Employing Signal Representation in Multiple Transform Domains, Digital Signal Processing11 (2001) 359–370Vector quantization schemes are widely used for waveform coding of one- and multidimensional signals. In this contribution, a novel energy-based, split vector quantization technique is presented, which represents digital signals efficiently as measured by the number of bits per sample for a predetermined signal reconstruction quality. In this approach, each signal vector is projected into multiple transform domains. In the learning mode, for a given transform domain representation, the transformed vector is split into subvectors (subbands) of equal average energy estimated from the transformed training vector ensemble. An equal number of bits is assigned to each subvector. A codebook is then designed for each equal energy subband of each transform domain representation. In the running mode, the coder selects codes from the domain that best represents the signal vector. The proposed multiple transform, split vector quantizer is developed and its performance is evaluated for both single-stage and multistage implementations. Several single transform vector quantizers for waveform coding exist, some of which employ energy-based bit allocation. Sample results using one-dimensional speech signals confirm the superior performance of the proposed scheme over existing single transform vector quantizers for waveform coding.     

Fuzzy declustering-based vector quantization

Vector quantization is a useful approach for multi-dimensional data compression and pattern classification. One of the most popular techniques for vector quantization design is the LBG (Linde, Buzo, Gray) algorithm. To address the problem of producing poor estimate of vector centroids which are subjected to biased data in vector quantization; we propose a fuzzy declustering strategy for the LBG algorithm. The proposed technique calculates appropriate declustering weights to adjust the global data distribution. Using the result of fuzzy declustering-based vector quantization design, we incorporate the notion of fuzzy partition entropy into the distortion measures that can be useful for classification of spectral features. Experimental results obtained from simulated and real data sets demonstrate the effective performance of the proposed approach.     

A cluster centers initialization method for clustering categorical data

The leading partitional clustering technique, k-modes, is one of the most computationally efficient clustering methods for categorical data. However, the performance of the k-modes clustering algorithm which converges to numerous local minima strongly depends on initial cluster centers. Currently, most methods of initialization cluster centers are mainly for numerical data. Due to lack of geometry for the categorical data, these methods used in cluster centers initialization for numerical data are not applicable to categorical data. This paper proposes a novel initialization method for categorical data which is implemented to the k-modes algorithm. The method integrates the distance and the density together to select initial cluster centers and overcomes shortcomings of the existing initialization methods for categorical data. Experimental results illustrate the proposed initialization method is effective and can be applied to large data sets for its linear time complexity with respect to the number of data objects.     

Robust Adaptive Attitude Control for Non-rigid Spacecraft With Quantized Control Input

In this paper,an adaptive backstepping control scheme is proposed for attitude tracking of non-rigid spacecraft in the presence of input quantization,inertial uncertainty and external disturbance.TThe control signal for each actuator is quantized by sector-bounded quantizers,including the logarithmic quantizer and the hysteresis quantizer.By describing the impact of quantization in a new affine model and introducing a smooth function and a novel form of the control signal,the influence caused by input quantization and external disturbance is properly compensated for.Moreover,with the aid of the adaptive control technique,our approach can achieve attitude tracking without the explicit knowledge of inertial parameters.Unlike existing attitude control schemes for spacecraft,in this paper,the quantization parameters can be unknown,and the bounds of inertial parameters and disturbance are also not needed.In addition to proving the stability of the closed-loop system,the relationship between the control performance and design parameters is analyzed.Simulation results are presented to illustrate the effectiveness of the proposed scheme.     

Control under quantization, saturation and delay: An LMI approach

This paper studies quantized and delayed state-feedback control of linear systems with given constant bounds on the quantization error and on the time-varying delay. The quantizer is supposed to be saturated. We consider two types of quantizations: quantized control input and quantized state. The controller is designed with the following property: all the states of the closed-loop system starting from a neighborhood of the origin exponentially converge to some bounded region (both, in Rⁿ and in some infinite-dimensional state space). Under suitable conditions the attractive region is inside the initial one. We propose decomposition of the quantization into a sum of a saturation and of a uniformly bounded (by the quantization error bound) disturbance. A Linear Matrix Inequalities (LMIs) approach via Lyapunov-Krasovskii method originating in the earlier work [Fridman, E., Dambrine, M., & Yeganefar, N. (2008). On input-to-state stability of systems with time-delay: A matrix inequalities approach. Automatica, 44, 2364-2369] is extended to the case of saturated quantizer and of quantized state and is based on the simplified and improved Lyapunov-Krasovskii technique.     

应用多索引加法量化编码的近邻检索算法

目的 海量图像检索技术是计算机视觉领域研究热点之一,一个基本的思路是对数据库中所有图像提取特征,然后定义特征相似性度量,进行近邻检索。海量图像检索技术,关键的是设计满足存储需求和效率的近邻检索算法。为了提高图像视觉特征的近似表示精度和降低图像视觉特征的存储空间需求,提出了一种多索引加法量化方法。方法 由于线性搜索算法复杂度高,而且为了满足检索的实时性,需把图像描述符存储在内存中,不能满足大规模检索系统的需求。基于非线性检索的优越性,本文对非穷尽搜索的多索引结构和量化编码进行了探索新研究。利用多索引结构将原始数据空间划分成多个子空间,把每个子空间数据项分配到不同的倒排列表中,然后使用压缩编码的加法量化方法编码倒排列表中的残差数据项,进一步减少对原始空间的量化损失。在近邻检索时采用非穷尽搜索的策略,只在少数倒排列表中检索近邻项,可以大大减少检索时间成本,而且检索过程中不用存储原始数据,只需存储数据集中每个数据项在加法量化码书中的码字索引,大大减少内存消耗。结果 为了验证算法的有效性,在3个数据集SIFT、GIST、MNIST上进行测试,召回率相比近几年算法提升4%~15%,平均查准率提高12%左右,检索时间与最快的算法持平。结论 本文提出的多索引加法量化编码算法,有效改善了图像视觉特征的近似表示精度和存储空间需求,并提升了在大规模数据集的检索准确率和召回率。本文算法主要针对特征进行近邻检索,适用于海量图像以及其他多媒体数据的近邻检索。     

Model Free Adaptive Control for a Class of Nonlinear Systems Using Quantized Information

This paper considers the stability of model free adaptive control systems with quantized information. Two quantized model free adaptive control (QMFAC) algorithms are proposed by using different signal quantization schemes, and here the logarithmic quantizer is introduced to decode these signals with a number of quantization levels. By using the sector bound method, the stability conditions of proposed QMFAC algorithms can be given and the effect of quantization error for such systems can also be discussed. It is shown that the tracking error under the QMFAC algorithm with system output quantized signal is proved to converge to a bound, and the bound depending on quantization density and desired trajectory. Thus, the tracking error under the QMFAC algorithm with tracking error quantized signal converges to zero. Two illustrative examples are given to validate the theoretical results.