Adaptive nearest neighbor pattern classification

A variant of nearest-neighbor (NN) pattern classification and supervised learning by learning vector quantization (LVQ) is described. The decision surface mapping method (DSM) is a fast supervised learning algorithm and is a member of the LVQ family of algorithms. A relatively small number of prototypes are selected from a training set of correctly classified samples. The training set is then used to adapt these prototypes to map the decision surface separating the classes. This algorithm is compared with NN pattern classification, learning vector quantization, and a two-layer perceptron trained by error backpropagation. When the class boundaries are sharply defined (i.e., no classification error in the training set), the DSM algorithm outperforms these methods with respect to error rates, learning rates, and the number of prototypes required to describe class boundaries.     

学习矢量量化的推广及其典型形式的比较

无监督学习矢量量化(LVQ)是一类基于最小化风险函数的聚类方法,文中通过对无监督LVQ风险函数的研究,提出了无监督LVQ算法的广义形式,在此基础上将当前典型的LVQ算法表示为基于不同尺度函数的LVQ算法,极大地方便了学习矢量量化神经网络的推广与应用。通过对无监督LVQ神经网络的改造,得到了基于无监督聚类算法的有监督LVQ神经网络,并将其应用于说话人辨认,取得了满意的结果并比较了几种典型聚类算法的优劣。     

A novel feature selection algorithm based on LVQ hypothesis margin

Feature selection has been widely discussed as an important preprocessing step in machine learning and data mining. In this paper, a new feature selection evaluation criterion based on low-loss learning vector quantization (LVQ) classification is proposed. Based on the evaluation criterion, a feature selection algorithm that optimizes the hypothesis margin of LVQ classification through minimizing its loss function is presented. Some experiments that are compared with well-known SVM-RFE and Relief are carried out on 4 UCI data sets using Naive Bayes and RBF Network classifier. Experimental results show that new algorithm achieves similar or even higher performance than Relief on all training data and has better or comparable performance than SVM-RFE.     

A Batch Learning Vector Quantization Algorithm for Nearest Neighbour Classification

We introduce a batch learning algorithm to design the set of prototypes of 1 nearest-neighbour classifiers. Like Kohonen's LVQ algorithms, this procedure tends to perform vector quantization over a probability density function that has zero points at Bayes borders. Although it differs significantly from their online counterparts since: (1) its statistical goal is clearer and better defined; and (2) it converges superlinearly due to its use of the very fast Newton's optimization method. Experiments results using artificial data confirm faster training time and better classification performance than Kohonen's LVQ algorithms.     

An axiomatic approach to soft learning vector quantization andclustering

This paper presents an axiomatic approach to soft learning vector quantization (LVQ) and clustering based on reformulation. The reformulation of the fuzzy c-means (FCM) algorithm provides the basis for reformulating entropy-constrained fuzzy clustering (ECFC) algorithms. According to the proposed approach, the development of specific algorithms reduces to the selection of a generator function. Linear generator functions lead to the FCM and fuzzy learning vector quantization algorithms while exponential generator functions lead to ECFC and entropy-constrained learning vector quantization algorithms. The reformulation of LVQ and clustering algorithms also provides the basis for developing uncertainty measures that can identify feature vectors equidistant from all prototypes. These measures are employed by a procedure developed to make soft LVQ and clustering algorithms capable of identifying outliers in the data set. This procedure is evaluated by testing the algorithms generated by linear and exponential generator functions on speech data.     

Soft learning vector quantization

Learning vector quantization (LVQ) is a popular class of adaptive nearest prototype classifiers for multiclass classification, but learning algorithms from this family have so far been proposed on heuristic grounds. Here, we take a more principled approach and derive two variants of LVQ using a gaussian mixture ansatz. We propose an objective function based on a likelihood ratio and derive a learning rule using gradient descent. The new approach provides a way to extend the algorithms of the LVQ family to different distance measure and allows for the design of "soft" LVQ algorithms. Benchmark results show that the new methods lead to better classification performance than LVQ 2.1. An additional benefit of the new method is that model assumptions are made explicit, so that the method can be adapted more easily to different kinds of problems.     

A new algorithm for Kohonen layer learning with application topower system stability analysis

In certain classification problems, input patterns are not distributed in a clustering manner but distributed uniformly in an input space and there exist certain critical hyperplanes called decision boundaries. Since learning vector quantization (LVQ) classifies an input vector based on the nearest neighbor, the codebook vectors away from the decision boundaries are redundant. This paper presents an alternative algorithm called boundary search algorithm (BSA) for the purpose of solving this redundancy problem. The BSA finds a fixed number of codebook vectors near decision boundaries by selecting appropriate training vectors. It is found to be more efficient compared with LVQ and its validity is demonstrated with satisfaction in the transient stability analysis of a power system.     

Initialization insensitive LVQ algorithm based on cost-function adaptation

A learning vector quantization (LVQ) algorithm called harmonic to minimum LVQ algorithm (H2M-LVQ)¹ is presented to tackle the initialization sensitiveness problem associated with the original generalized LVQ (GLVQ) algorithm. Experimental results show superior performance of the H2M-LVQ algorithm over the GLVQ and one of its variants on several datasets.     

Repairs to GLVQ: a new family of competitive learning schemes

First, we identify an algorithmic defect of the generalized learning vector quantization (GLVQ) scheme that causes it to behave erratically for a certain scaling of the input data. We show that GLVQ can behave incorrectly because its learning rates are reciprocally dependent on the sum of squares of distances from an input vector to the node weight vectors. Finally, we propose a new family of models-the GLVQ-F family-that remedies the problem. We derive competitive learning algorithms for each member of the GLVQ-F model and prove that they are invariant to all scalings of the data. We show that GLVQ-F offers a wide range of learning models since it reduces to LVQ as its weighting exponent (a parameter of the algorithm) approaches one from above. As this parameter increases, GLVQ-F then transitions to a model in which either all nodes may be excited according to their (inverse) distances from an input or in which the winner is excited while losers are penalized. And as this parameter increases without limit, GLVQ-F updates all nodes equally. We illustrate the failure of GLVQ and success of GLVQ-F with the IRIS data.     

A particle swarm optimization based simultaneous learning framework for clustering and classification

A particle swarm optimization based simultaneous learning framework for clustering and classification (PSOSLCC) is proposed in this paper. Firstly, an improved particle swarm optimization (PSO) is used to partition the training samples, the number of clusters must be given in advance, an automatic clustering algorithm rather than the trial and error is adopted to find the proper number of clusters, and a set of clustering centers is obtained to form classification mechanism. Secondly, in order to exploit more useful local information and get a better optimizing result, a global factor is introduced to the update strategy update strategy of particle in PSO. PSOSLCC has been extensively compared with fuzzy relational classifier (FRC), vector quantization and learning vector quantization (VQ+LVQ3), and radial basis function neural network (RBFNN), a simultaneous learning framework for clustering and classification (SCC) over several real-life datasets, the experimental results indicate that the proposed algorithm not only greatly reduces the time complexity, but also obtains better classification accuracy for most datasets used in this paper. Moreover, PSOSLCC is applied to a real world application, namely texture image segmentation with a good performance obtained, which shows that the proposed algorithm has a potential of classifying the problems with large scale.     

基于AGA-LVQ神经网络的软件可靠性预测模型研究

针对当前大多数软件可靠性预测模型预测准确率不高等问题,利用LVQ神经网络的非线性运算能力和自适应遗传算法(AGA)的参数寻优能力,提出了一种基于AGA-LVQ的软件可靠性预测模型。首先对待预测的数据用主成分分析(PCA)等方法进行预处理以降低维度,去除冗余和错误数据,然后根据自适应遗传算法来计算最优的LVQ神经网络初始权值向量,最后运用LVQ神经网络进行软件可靠性预测实验。通过与传统方法的对比,证明该方法具有较高的预测准确率。     

Alternative learning vector quantization

In this paper, we discuss the influence of feature vectors contributions at each learning time t on a sequential-type competitive learning algorithm. We then give a learning rate annealing schedule to improve the unsupervised learning vector quantization (ULVQ) algorithm which uses the winner-take-all competitive learning principle in the self-organizing map (SOM). We also discuss the noisy and outlying problems of a sequential competitive learning algorithm and then propose an alternative learning formula to make the sequential competitive learning robust to noise and outliers. Combining the proposed learning rate annealing schedule and alternative learning formula, we propose an alternative learning vector quantization (ALVQ) algorithm. Some discussion and experimental results from comparing ALVQ with ULVQ show the superiority of the proposed method.     

Prototype-based minimum classification error/generalized probabilistic descent training for various speech units

In previous work we reported high classification rates for learning vector quantization (LVQ) networks trained to classify phoneme tokens shifted in time. It has since been shown that the framework of minimum classification error (MCE) and generalized probabilistic descent (GPD) can treat LVQ as a special case of a general method for gradient descent on a rigorously defined classification loss measure that closely reflects the misclassification rate. This framework allows us to extend LVQ into a prototype-based minimum error classifier (PBMEC) appropriate for the classification of various speech units which the original LVQ was unable to treat. Speech categories are represented using a prototype-based multi-state architecture incorporating a dynamic time warping procedure. We present results for the difficult E-set task, as well as for isolated word recognition for a vocabulary of 5240 words, that reveal clear gains in performance as a result of using PBMEC. In addition, we discuss the issue of smoothing the loss function from the perspective of increasing classifier robustness.     

基于WT和LVQ网络的多姿态人脸识别

提出了基于小波变换和学习矢量量化网络相结合的新方法进行人脸识别。小波变换具有良好的多尺度特征表达能力，能将图像的大部分能量集中到最低分辨率子图像，可以很好地对图像降维和表征人脸图像的特征。LVQ算法是在有教师状态下对竞争层进行训练的一种学习算法。LVQ网络结构简单，但却表现出比BP网络更强的有效性和鲁棒性。实验表明该方法对表情和姿态变化的人脸具有良好的分类性能和识别效率。     

Time integration and reject options for probabilistic output of pairwise LVQ

Learning vector quantization (LVQ) constitutes a very popular machine learning technology with applications, for example, in biomedical data analysis, predictive maintenance/quality as well as product individualization. Albeit probabilistic LVQ variants exist, its deterministic counterparts are often preferred due to their better efficiency. The latter do not allow an immediate probabilistic interpretation of its output; hence, a rejection of classification based on confidence values is not possible. In this contribution, we investigate different schemes how to extend and integrate pairwise LVQ schemes to an overall probabilistic output, in comparison with a recent heuristic surrogate measure for the security of the classification, which is directly based on LVQ’s multi-class classification scheme. Furthermore, we propose a canonic way how to fuse these values over a given time window in case a possibly disrupted measurement is taken over a longer time interval to counter the uncertainty of a single point in time. Experimental results indicate that an explicit probabilistic treatment often yields superior results as compared to a standard deterministic LVQ method, but metric learning is able to annul this difference. Fusion over a short time period is beneficial in case of an unclear classification.

     

Extraction rice-planted areas by RADARSAT data using neural networks

A classification technique using the neural networks has recently been developed. We apply a neural network of learning vector quantization (LVQ) to classify remote-sensing data, including microwave and optical sensors, for the estimation of a rice-planted area. The method has the capability of nonlinear discrimination, and the classification function is determined by learning. The satellite data were observed before and after planting rice in 1999. Three sets of RADARSAT and one set of SPOT/HRV data were used in Higashi–Hiroshima, Japan. Three RADARSAT images from April to June were used for this study. The LVQ classification was applied the RADARSAT and SPOT to evaluate the estimate of the area of planted-rice. The results show that the true production rate of the rice-planted area estimation of RADASAT by LVQ was approximately 60% compared with that of SPOT by LVQ. It is shown that the present method is much better than the SAR image classification by the maximum likelihood method.     

Learning vector quantization for the probabilistic neural network

A modified version of the PNN (probabilistic neural network) learning phase which allows a considerable simplification of network structure by including a vector quantization of learning data is proposed. It can be useful if large training sets are available. The procedure has been successfully tested in two synthetic data experiments. The proposed network has been shown to improve the classification performance of the LVQ (learning vector quantization) procedure.     

Dictionary learning for VQ feature extraction in ECG beats classification

Vector quantization(VQ) can perform efficient feature extraction from electrocardiogram (ECG) with the advantages of dimensionality reduction and accuracy increase. However, the existing dictionary learning algorithms for vector quantization are sensitive to dirty data, which compromises the classification accuracy. To tackle the problem, we propose a novel dictionary learning algorithm that employs k-medoids cluster optimized by k-means++ and builds dictionaries by searching and using representative samples, which can avoid the interference of dirty data, and thus boost the classification performance of ECG systems based on vector quantization features. We apply our algorithm to vector quantization feature extraction for ECG beats classification, and compare it with popular features such as sampling point feature, fast Fourier transform feature, discrete wavelet transform feature, and with our previous beats vector quantization feature. The results show that the proposed method yields the highest accuracy and is capable of reducing the computational complexity of ECG beats classification system. The proposed dictionary learning algorithm provides more efficient encoding for ECG beats, and can improve ECG classification systems based on encoded feature.