Soft learning vector quantization and clustering algorithms based on non-Euclidean norms: multinorm algorithms

This paper presents the development of soft clustering and learning vector quantization (LVQ) algorithms that rely on multiple weighted norms to measure the distance between the feature vectors and their prototypes. Clustering and LVQ are formulated in this paper as the minimization of a reformulation function that employs distinct weighted norms to measure the distance between each of the prototypes and the feature vectors under a set of equality constraints imposed on the weight matrices. Fuzzy LVQ and clustering algorithms are obtained as special cases of the proposed formulation. The resulting clustering algorithm is evaluated and benchmarked on three data sets that differ in terms of the data structure and the dimensionality of the feature vectors. This experimental evaluation indicates that the proposed multinorm algorithm outperforms algorithms employing the Euclidean norm as well as existing clustering algorithms employing weighted norms.     

Soft learning vector quantization and clustering algorithms basedon ordered weighted aggregation operators

This paper presents the development and investigates the properties of ordered weighted learning vector quantization (LVQ) and clustering algorithms. These algorithms are developed by using gradient descent to minimize reformulation functions based on aggregation operators. An axiomatic approach provides conditions for selecting aggregation operators that lead to admissible reformulation functions. Minimization of admissible reformulation functions based on ordered weighted aggregation operators produces a family of soft LVQ and clustering algorithms, which includes fuzzy LVQ and clustering algorithms as special cases. The proposed LVQ and clustering algorithms are used to perform segmentation of magnetic resonance (MR) images of the brain. The diagnostic value of the segmented MR images provides the basis for evaluating a variety of ordered weighted LVQ and clustering algorithms.     

Soft learning vector quantization

Learning vector quantization (LVQ) is a popular class of adaptive nearest prototype classifiers for multiclass classification, but learning algorithms from this family have so far been proposed on heuristic grounds. Here, we take a more principled approach and derive two variants of LVQ using a gaussian mixture ansatz. We propose an objective function based on a likelihood ratio and derive a learning rule using gradient descent. The new approach provides a way to extend the algorithms of the LVQ family to different distance measure and allows for the design of "soft" LVQ algorithms. Benchmark results show that the new methods lead to better classification performance than LVQ 2.1. An additional benefit of the new method is that model assumptions are made explicit, so that the method can be adapted more easily to different kinds of problems.     

Fuzzy algorithms for learning vector quantization

This paper presents the development of fuzzy algorithms for learning vector quantization (FALVQ). These algorithms are derived by minimizing the weighted sum of the squared Euclidean distances between an input vector, which represents a feature vector, and the weight vectors of a competitive learning vector quantization (LVQ) network, which represent the prototypes. This formulation leads to competitive algorithms, which allow each input vector to attract all prototypes. The strength of attraction between each input and the prototypes is determined by a set of membership functions, which can be selected on the basis of specific criteria. A gradient-descent-based learning rule is derived for a general class of admissible membership functions which satisfy certain properties. The FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms are developed by selecting admissible membership functions with different properties. The proposed algorithms are tested and evaluated using the IRIS data set. The efficiency of the proposed algorithms is also illustrated by their use in codebook design required for image compression based on vector quantization.     

Constrained-storage multistage vector quantization based on genetic algorithms

Multistage vector quantization (MSVQ) and their variants have been recently proposed. Before MSVQ is designed, the user must artificially determine the number of codewords in each VQ stage. However, the users usually have no idea regarding the number of codewords in each VQ stage, and thus doubt whether the resulting MSVQ is optimal. This paper proposes the genetic design (GD) algorithm to design the MSVQ. The GD algorithm can automatically find the number of codewords to optimize each VQ stage according to the rate–distortion performance. Thus, the MSVQ based on the GD algorithm, namely MSVQ(GD), is proposed here. Furthermore, using a sharing codebook (SC) can further reduce the storage size of MSVQ. Combining numerous similar codewords in the VQ stages of MSVQ produces the codewords of the sharing codebook. This paper proposes the genetic merge (GM) algorithm to design the SC of MSVQ. Therefore, the constrained-storage MSVQ using a SC, namely CSMSVQ, is proposed and outperforms other MSVQs in the experiments presented here.     

Unsupervised learning with normalised data and non-Euclidean norms

The measurement of distance is one of the key steps in the unsupervised learning process, as it is through these distance measurements that patterns and correlations are discovered. We examined the characteristics of both non-Euclidean norms and data normalisation within the unsupervised learning environment. We empirically assessed the performance of the K-means, neural gas, growing neural gas and self-organising map algorithms with a range of real-world data sets and concluded that data normalisation is both beneficial in learning class structure and in reducing the unpredictable influence of the norm.     

An integrated approach to fuzzy learning vector quantization andfuzzy c-means clustering

Derives an interpretation for a family of competitive learning algorithms and investigates their relationship to fuzzy c-means and fuzzy learning vector quantization. These algorithms map a set of feature vectors into a set of prototypes associated with a competitive network that performs unsupervised learning. Derivation of the new algorithms is accomplished by minimizing an average generalized distance between the feature vectors and prototypes using gradient descent. A close relationship between the resulting algorithms and fuzzy c-means is revealed by investigating the functionals involved. It is also shown that the fuzzy c-means and fuzzy learning vector quantization algorithms are related to the proposed algorithms if the learning rate at each iteration is selected to satisfy a certain condition     

Alternative learning vector quantization

In this paper, we discuss the influence of feature vectors contributions at each learning time t on a sequential-type competitive learning algorithm. We then give a learning rate annealing schedule to improve the unsupervised learning vector quantization (ULVQ) algorithm which uses the winner-take-all competitive learning principle in the self-organizing map (SOM). We also discuss the noisy and outlying problems of a sequential competitive learning algorithm and then propose an alternative learning formula to make the sequential competitive learning robust to noise and outliers. Combining the proposed learning rate annealing schedule and alternative learning formula, we propose an alternative learning vector quantization (ALVQ) algorithm. Some discussion and experimental results from comparing ALVQ with ULVQ show the superiority of the proposed method.     

基于学习向量量化神经网络的软件可靠性预测

针对传统的软件可靠性预测模型在实际应用中存在预测泛化性能不佳等问题,提出一种基于学习向量量化（LVQ）神经网络的软件可靠性预测模型。首先分析了LVQ神经网络的结构特点以及它与软件可靠性预测的联系,然后运用该网络来进行软件可靠性的预测,并基于美国国家航空航天局（NASA）软件数据项目中的实例数据集,运用Matlab工具进行了仿真实验。通过与传统预测方法的对比,证明该方法具有可行性和较高的预测泛化性能。     

Online learning vector quantization: a harmonic competitionapproach based on conservation network

This paper presents a self-creating neural network in which a conservation principle is incorporated with the competitive learning algorithm to harmonize equi-probable and equi-distortion criteria. Each node is associated with a measure of vitality which is updated after each input presentation. The total amount of vitality in the network at any time is 1, hence the name conservation. Competitive learning based on a vitality conservation principle is near-optimum, in the sense that problem of trapping in a local minimum is alleviated by adding perturbations to the learning rate during node generation processes. Combined with a procedure that redistributes the learning rate variables after generation and removal of nodes, the competitive conservation strategy provides a novel approach to the problem of harmonizing equi-error and equi-probable criteria. The training process is smooth and incremental, it not only achieves the biologically plausible learning property, but also facilitates systematic derivations for training parameters. Comparison studies on learning vector quantization involving stationary and nonstationary, structured and nonstructured inputs demonstrate that the proposed network outperforms other competitive networks in terms of quantization error, learning speed, and codeword search efficiency.     

基于提升小波变换与学习矢量量化网络的鉴别分析方法

提出了一种基于提升小波变换（LWT）和学习矢量量化网络（LVQ）相结合的鉴别分析方法。提升小波又叫作第二代小波,比传统的第一代小波变换更为快速有效,利用它的多分辨率特性,可以获取输入图像的低频信息并使图像降维。LVQ算法是在有教师状态下对竞争层进行训练的一种学习算法。LVQ网络结构简单,但却表现出比BP网络更强的有效性和鲁棒性。在ORL标准人脸库及现实人脸图像上的实验结果表明该方法具有很好的鉴别分析能力。     

基于演进向量量化聚类的增量模糊关联分类方法

为了提高动态数据集上模糊关联分类器（FAC）的建模效率,提出了一种基于演进向量量化（eVQ）聚类的增量模糊关联分类方法。首先,采用eVQ聚类算法增量更新数量属性上的高斯隶属度函数参数;然后,扩展早剪枝更新（UWEP）算法,使之适用于增量挖掘模糊频繁项;最后,以模糊相关度（FCORR）和分类规则前件长度为度量方式裁剪并更新模糊关联分类规则库。在4个UCI标准数据集上的实验结果表明,与批量模糊关联分类建模方法相比,所提方法能够在保证分类精度和解释性的前提下,减少模糊关联分类器的训练时间;基于eVQ的高斯隶属度函数的增量更新有助于提高动态数据集上模糊关联分类器的分类精度。     

Adaptive data hiding for vector quantization images based on overlapping codeword clustering

In this paper, an overlapping codeword clustering based data hiding scheme is presented. In this scheme, a mapping table is designed to determine the overlapping codeword clustering and to indicate the index modification in the secret embedding. The mapping table explores the relationship among the sub-codebook’s size, the codeword’s order and the embedding secret message to which the codeword overlapping in sub-codebooks with different sizes is permitted. In addition, the secret embedding is also determined according to the mapping table.The experimental results showed that the number of partitioned sub-codebooks was increased significantly. The average hiding capacity was about 30 K bits while the average embedding distortion was about 1.2 dB. In comparison to similar methods, the proposed scheme provided a larger hiding capacity than others while preserving a similar stego-image quality. Furthermore, the proposed scheme offered a better proportion of hiding compared to image distortion.     

Extensions of vector quantization for incremental clustering

In this paper, we extend the conventional vector quantization by incorporating a vigilance parameter, which steers the tradeoff between plasticity and stability during incremental online learning. This is motivated in the adaptive resonance theory (ART) network approach and is exploited in our paper for forming a one-pass incremental and evolving variant of vector quantization. This variant can be applied for online clustering, classification and approximation tasks with an unknown number of clusters. Additionally, two novel extensions are described: one concerns the incorporation of the sphere of influence of clusters in the vector quantization learning process by selecting the ‘winning cluster’ based on the distances of a data point to the surface of all clusters. Another one introduces a deletion of cluster satellites and an online split-and-merge strategy: clusters are dynamically split and merged after each incremental learning step. Both strategies prevent the algorithm to generate a wrong cluster partition due to a bad a priori setting of the most essential parameter(s). The extensions will be applied to clustering of two- and high-dimensional data, within an image classification framework and for model-based fault detection based on data-driven evolving fuzzy models.     

基于模糊核LVQ的Sammon非线性映射算法

提出了一种基于可靠稳定的模糊核学习矢量量化（FKLVQ）聚类的Sammon非线性映射新算法。该方法通过Mercer核，将数据空间映射到高维特征空间，并在此特征空间上进行FKLVQ学习获取数据空间有效且稳定的聚类权矢量，然后在特征空间和输出空间上仅针对各空间的数据样本和它们各自的聚类权矢量进行Sammon非线性核映射。这样既降低了计算的复杂度，又使数据空间和输出空间上数据点与聚类中心间的距离信息保持相似。仿真结果验证了该方法的可靠性和稳定性。     

基于学习矢量量化的空调压缩机声纹诊断方法

针对传统空调压缩机故障诊断工况信号采集困难的缺点,提出一种基于学习矢量量化(learning vector quantiza-tion,LVQ)神经网络的空调压缩机声纹识别模型用于空调压缩机故障诊断,将声纹识别技术引入压缩机故障诊断.对压缩机的声音数据进行预处理,包括预加重、分帧、加窗,在分帧步骤中针对压缩机的声音特性...     

Divergence-based classification in learning vector quantization

We discuss the use of divergences in dissimilarity-based classification. Divergences can be employed whenever vectorial data consists of non-negative, potentially normalized features. This is, for instance, the case in spectral data or histograms. In particular, we introduce and study divergence based learning vector quantization (DLVQ). We derive cost function based DLVQ schemes for the family of γ‐divergences which includes the well-known Kullback-Leibler divergence and the so-called Cauchy-Schwarz divergence as special cases. The corresponding training schemes are applied to two different real world data sets. The first one, a benchmark data set (Wisconsin Breast Cancer) is available in the public domain. In the second problem, color histograms of leaf images are used to detect the presence of cassava mosaic disease in cassava plants. We compare the use of standard Euclidean distances with DLVQ for different parameter settings. We show that DLVQ can yield superior classification accuracies and Receiver Operating Characteristics.     

基于矢量量化和深度优先搜索的轨迹分布模式学习算法

在对轨迹流矢量进行量化编码的基础上,提出了一种基于深度优先搜索的轨迹分布模式提取算法,生成了能够描述轨迹分布的序列模式图,并给出了与之相应的异常检测和行为预测方法。对不同场景的可见光和红外序列图像的实验表明,本文方法不仅能够学习轨迹中流矢量的分布,而且能够反映它们之间的时序关系,可以应用于室外复杂场景的目标异常行为检测。     

基于多种聚类算法和多元线性回归的多分类主动学习算法

针对传统岩性识别方法识别精度低,难以和地质经验有机结合的问题,提出了一种基于多种聚类算法和多元线性回归的多分类主动学习算法（ALCL）。首先,通过多种异构聚类算法聚类得到对应每种算法的类别矩阵,并通过查询公共点对类别矩阵进行标记和预分类;其次,提出优先级最大搜寻策略和最混乱查询策略选取用于训练聚类算法权重系数模型的关键实例;然后,定义目标求解函数,通过训练关键实例求解得到每种聚类算法的权重系数;最后,结合权重系数进行分类计算,从而对结果置信度高的样本进行分类。应用大庆油田油井的6个公开岩性数据集进行实验,实验结果表明,ALCL的分类精度最高时,比传统监督学习算法和其他主动学习算法提高了2.07%~14.01%。假设检验和显著性分析的结果验证了ALCL在岩性识别问题上具有更好的分类效果。