Soft learning vector quantization and clustering algorithms based on non-Euclidean norms: multinorm algorithms

This paper presents the development of soft clustering and learning vector quantization (LVQ) algorithms that rely on multiple weighted norms to measure the distance between the feature vectors and their prototypes. Clustering and LVQ are formulated in this paper as the minimization of a reformulation function that employs distinct weighted norms to measure the distance between each of the prototypes and the feature vectors under a set of equality constraints imposed on the weight matrices. Fuzzy LVQ and clustering algorithms are obtained as special cases of the proposed formulation. The resulting clustering algorithm is evaluated and benchmarked on three data sets that differ in terms of the data structure and the dimensionality of the feature vectors. This experimental evaluation indicates that the proposed multinorm algorithm outperforms algorithms employing the Euclidean norm as well as existing clustering algorithms employing weighted norms.     

Soft learning vector quantization and clustering algorithms based on non-Euclidean norms: single-norm algorithms

This paper presents the development of soft clustering and learning vector quantization (LVQ) algorithms that rely on a weighted norm to measure the distance between the feature vectors and their prototypes. The development of LVQ and clustering algorithms is based on the minimization of a reformulation function under the constraint that the generalized mean of the norm weights be constant. According to the proposed formulation, the norm weights can be computed from the data in an iterative fashion together with the prototypes. An error analysis provides some guidelines for selecting the parameter involved in the definition of the generalized mean in terms of the feature variances. The algorithms produced from this formulation are easy to implement and they are almost as fast as clustering algorithms relying on the Euclidean norm. An experimental evaluation on four data sets indicates that the proposed algorithms outperform consistently clustering algorithms relying on the Euclidean norm and they are strong competitors to non-Euclidean algorithms which are computationally more demanding.     

Soft learning vector quantization

Learning vector quantization (LVQ) is a popular class of adaptive nearest prototype classifiers for multiclass classification, but learning algorithms from this family have so far been proposed on heuristic grounds. Here, we take a more principled approach and derive two variants of LVQ using a gaussian mixture ansatz. We propose an objective function based on a likelihood ratio and derive a learning rule using gradient descent. The new approach provides a way to extend the algorithms of the LVQ family to different distance measure and allows for the design of "soft" LVQ algorithms. Benchmark results show that the new methods lead to better classification performance than LVQ 2.1. An additional benefit of the new method is that model assumptions are made explicit, so that the method can be adapted more easily to different kinds of problems.     

Dependent uncertain ordered weighted aggregation operators

Xu and Da [Z.S. Xu, Q.L. Da, The uncertain OWA operator, International Journal of Intelligent Systems, 17 (2002) 569–575] introduced the uncertain ordered weighted averaging (UOWA) operator to aggregate the input arguments taking the form of intervals rather than exact numbers. In this paper, we develop some dependent uncertain ordered weighted aggregation operators, including dependent uncertain ordered weighted averaging (DUOWA) operators and dependent uncertain ordered weighted geometric (DUOWG) operators, in which the associated weights only depend on the aggregated interval arguments and can relieve the influence of unfair interval arguments on the aggregated results by assigning low weights to those “false” and “biased” ones.     

Variance measures with ordered weighted aggregation operators

The variance is a statistical measure widely used in many real-life application areas. This article makes an extensive investigation on variance measure in the case when the uncertainty is not of a probabilistic nature. It generalizes the notion of variance as well as the theory of ordered weighted aggregation operators. First, we extend the idea of representative value/expected value of a decision maker and develop some new deviation measures based on ordered weighted geometric (OWG) average and ordered weighted harmonic average (OWHA) operators. These measures are developed with the consideration that decision maker can represent his/her attitudinal expected value by using any one of the ordered weighted aggregation (OWA) operators. Further, this study proposes some deviation measures by using the generalized-OWA (GOWA) and Quasi-OWA as an expected value of decision maker and discusses their particular cases. Second, a number of generalized deviation measures are introduced by taking the generalized arithmetic mean and quasi-arithmetic means for aggregation of the individual dispersion. This approach provides an ability to the user for considering the deviation under different realistic-scenario. These measures lead to many interesting particular and limiting cases and enrich the use of ordered weighted aggregation operators in the variance.     

Fuzzy algorithms for learning vector quantization

This paper presents the development of fuzzy algorithms for learning vector quantization (FALVQ). These algorithms are derived by minimizing the weighted sum of the squared Euclidean distances between an input vector, which represents a feature vector, and the weight vectors of a competitive learning vector quantization (LVQ) network, which represent the prototypes. This formulation leads to competitive algorithms, which allow each input vector to attract all prototypes. The strength of attraction between each input and the prototypes is determined by a set of membership functions, which can be selected on the basis of specific criteria. A gradient-descent-based learning rule is derived for a general class of admissible membership functions which satisfy certain properties. The FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms are developed by selecting admissible membership functions with different properties. The proposed algorithms are tested and evaluated using the IRIS data set. The efficiency of the proposed algorithms is also illustrated by their use in codebook design required for image compression based on vector quantization.     

Soft clustering using weighted one-class support vector machines

This paper describes a new soft clustering algorithm in which each cluster is modelled by a one-class support vector machine (OC-SVM). The proposed algorithm extends a previously proposed hard clustering algorithm, also based on OC-SVM representation of clusters. The key building block of our method is the weighted OC-SVM (WOC-SVM), a novel tool introduced in this paper, based on which an expectation-maximization-type soft clustering algorithm is defined. A deterministic annealing version of the algorithm is also introduced, and shown to improve the robustness with respect to initialization. Experimental results show that the proposed soft clustering algorithm outperforms its hard clustering counterpart, namely in terms of robustness with respect to initialization, as well as several other state-of-the-art methods.     

Generalized ordered weighted logarithm aggregation operators and their applications to group decision making

We present the generalized ordered weighted logarithm averaging (GOWLA) operator based on an optimal deviation model. It is a new aggregation operator that generalizes the ordered weighted geometric averaging (OWGA) operator. This operator adds to the OWGA operator an additional parameter. controlling the power to which the arguments are raised. We further generalize the GOWLA operator and obtain the generalized ordered weighted hybrid logarithm averaging (GOWHLA) operator. We next introduce a nonlinear objective programming model for determining GOWHLA weights and an approach to group decision making based on the GOWHLA operator. Finally, we present a numerical example to illustrate the new approach in human resource management problem. © 2010 Wiley Periodicals, Inc.     

Induced ordered weighted averaging operators

We briefly describe the Ordered Weighted Averaging (OWA) operator and discuss a methodology for learning the associated weighting vector from observational data. We then introduce a more general type of OWA operator called the Induced Ordered Weighted Averaging (IOWA) Operator. These operators take as their argument pairs, called OWA pairs, in which one component is used to induce an ordering over the second components which are then aggregated. A number of different aggregation situations have been shown to be representable in this framework. We then show how this tool can be used to represent different types of aggregation models.     

An integrated approach to fuzzy learning vector quantization andfuzzy c-means clustering

Derives an interpretation for a family of competitive learning algorithms and investigates their relationship to fuzzy c-means and fuzzy learning vector quantization. These algorithms map a set of feature vectors into a set of prototypes associated with a competitive network that performs unsupervised learning. Derivation of the new algorithms is accomplished by minimizing an average generalized distance between the feature vectors and prototypes using gradient descent. A close relationship between the resulting algorithms and fuzzy c-means is revealed by investigating the functionals involved. It is also shown that the fuzzy c-means and fuzzy learning vector quantization algorithms are related to the proposed algorithms if the learning rate at each iteration is selected to satisfy a certain condition     

Learning the stress function pattern of ordered weighted average aggregation using DBSCAN clustering

Ordered weighted average (OWA) operator provides a parameterized class of mean type operators between the minimum and the maximum. It is an important tool that can reflect the strategy of a decision maker for decision-making problems. In this study, the idea of obtaining the stress function from OWA weights has been put forward to generalize and characterize OWA weights. The main idea in this paper is mainly constructed on the basis that, generally, stress functions can be constructed using a mixture of constant and linear components. So, we can consider the stress function as a piecewise linear function. For obtaining stress functions as piecewise linear functions, we present a clustering-based approach for OWA weight generalization. This generalization is made using the DBSCAN algorithm as the learning method of a stress function associated with known OWA weights. In the learning process, the whole data set is divided into clusters, and then linear functions are obtained via a least squares estimator.     

Alternative learning vector quantization

In this paper, we discuss the influence of feature vectors contributions at each learning time t on a sequential-type competitive learning algorithm. We then give a learning rate annealing schedule to improve the unsupervised learning vector quantization (ULVQ) algorithm which uses the winner-take-all competitive learning principle in the self-organizing map (SOM). We also discuss the noisy and outlying problems of a sequential competitive learning algorithm and then propose an alternative learning formula to make the sequential competitive learning robust to noise and outliers. Combining the proposed learning rate annealing schedule and alternative learning formula, we propose an alternative learning vector quantization (ALVQ) algorithm. Some discussion and experimental results from comparing ALVQ with ULVQ show the superiority of the proposed method.     

The ordered weighted geometric averaging operators

The ordered weighted averaging (OWA) operator was introduced by Yager. 1 The fundamental aspect of the OWA operator is a reordering step in which the input arguments are rearranged in descending order. In this article, we propose two new classes of aggregation operators called ordered weighted geometric averaging (OWGA) operators and study some desired properties of these operators. Some methods for obtaining the associated weighting parameters are discussed, and the relationship between the OWA and DOWGA operators is also investigated. © 2002 Wiley Periodicals, Inc.     

Induced ordered weighted geometric operators and their use in the aggregation of multiplicative preference relations

In this article, we introduce the induced ordered weighted geometric (IOWG) operator and its properties. This is a more general type of OWG operator, which is based on the induced ordered weighted averaging (IOWA) operator. We provide some IOWG operators to aggregate multiplicative preference relations in group decision‐making (GDM) problems. In particular, we present the importance IOWG (I‐IOWG) operator, which induces the ordering of the argument values based on the importance of the information sources; the consistency IOWG (C‐IOWG) operator, which induces the ordering of the argument values based on the consistency of the information sources; and the preference IOWG (P‐IOWG) operator, which induces the ordering of the argument values based on the relative preference values associated with each one of them. We also provide a procedure to deal with “ties” regarding the ordering induced by the application of one of these IOWG operators. This procedure consists of a sequential application of the aforementioned IOWG operators. Finally, we analyze the reciprocity and consistency properties of the collective multiplicative preference relations obtained using IOWG operators. © 2004 Wiley Periodicals, Inc.     

Generalized compensative weighted averaging aggregation operators

The compensative weighted averaging (CWA) operator is generalized to develop a class of powerful generalized compensative weighted averaging (GCWA) operators with a very fine range of compensatory effects. The conventional means are shown to be the special cases of the proposed GCWA operator. A few extensions are investigated by combining GCWA operator with ordered weighted averaging (OWA) and induced OWA (IOWA) operators. An exponential class of aggregation operators such as exponential CWA, exponential OWA and exponential IOWA operators are developed. The usefulness of GCWA operators is shown through several examples and a case-study.     

Characterization of the ordered weighted averaging operators

This paper deals with the characterization of two classes of monotonic and neutral (MN) aggregation operators. The first class corresponds to (MN) aggregators which are stable for the same positive linear transformations and presents the ordered linkage property. The second class deals with (MN)-idempotent aggregators which are stable for positive linear transformations with the same unit, independent zeroes and ordered values. These two classes correspond to the weighted ordered averaging operator (OWA) introduced by Yager in 1988. It is also shown that the OWA aggregator can be expressed as a Choquet integral     

Similarity classifier with ordered weighted averaging operators

In this article we extend the similarity classifier to cover also ordered weighted averaging (OWA) operators. Earlier, similarity classifier was mainly used with generalized mean operator, but in this article we extend this aggregation process to cover more general OWA operators. With OWA operators we concentrate on linguistic quantifier guided aggregation where several different quantifiers are studied and on how they best suite for the similarity classifier. Our proposed method is applied to real world medical data sets which are new thyroid, hypothyroid, lymphography and hepatitis data sets. Results are very promising and show improvement compared to the earlier used generalized mean operator. In this article we will show that by using OWA operators instead of generalized mean, we can improve classification accuracy with chosen data sets.     

Extensions of vector quantization for incremental clustering

In this paper, we extend the conventional vector quantization by incorporating a vigilance parameter, which steers the tradeoff between plasticity and stability during incremental online learning. This is motivated in the adaptive resonance theory (ART) network approach and is exploited in our paper for forming a one-pass incremental and evolving variant of vector quantization. This variant can be applied for online clustering, classification and approximation tasks with an unknown number of clusters. Additionally, two novel extensions are described: one concerns the incorporation of the sphere of influence of clusters in the vector quantization learning process by selecting the ‘winning cluster’ based on the distances of a data point to the surface of all clusters. Another one introduces a deletion of cluster satellites and an online split-and-merge strategy: clusters are dynamically split and merged after each incremental learning step. Both strategies prevent the algorithm to generate a wrong cluster partition due to a bad a priori setting of the most essential parameter(s). The extensions will be applied to clustering of two- and high-dimensional data, within an image classification framework and for model-based fault detection based on data-driven evolving fuzzy models.     

Domination of ordered weighted averaging operators over t-norms

The fusion of transitive fuzzy relations preserving the transitivity is linked to the domination of the involved aggregation operator. The aim of this contribution is to investigate the domination of OWA operators over t-norms whereas the main emphasis is on the domination over the ukasiewicz t-norm. The domination of OWA operators and related operators over continuous Archimedean t-norms will also be discussed.This work was partly supported by network CEEPUS SK-42, COST Action 274 TARSKI and project APVT 20-023402.     

Ranking of alternatives with ordered weighted averaging operators

Multiattribute decision making is an important part of the decision process for both individual and group problems. We incorporate the fuzzy set theory and the basic nature of subjectivity due to ambiguity to achieve a flexible decision approach suitable for uncertain and fuzzy environments. Let us consider the analytic hierarchy process (AHP) in which the labels are structured as fuzzy numbers. To obtain the scoring that corresponds to the best alternative or the ranking of the alternatives, we need to use a total order for the fuzzy numbers involved in the problem. In this article, we consider a definition of such a total order, which is based on two subjective aspects: the degree of optimism/pessimism reflected with the ordered weighted averaging (OWA) operators. A numerical example is given to illustrate the approach. © 2004 Wiley Periodicals, Inc.