Accelerating fuzzy clustering

This paper extends earlier work [C. Borgelt, R. Kruse, Speeding up fuzzy clustering with neural network techniques, in: Proceedings of the 12th IEEE International Conference on Fuzzy Systems (FUZZ-IEEE’03, St. Louis, MO, USA), IEEE Press, Piscataway, NJ, USA, 2003] on an approach to accelerate fuzzy clustering by transferring methods that were originally developed to speed up the training process of (artificial) neural networks. The core idea is to consider the difference between two consecutive steps of the alternating optimization scheme of fuzzy clustering as providing a gradient. This “gradient” may then be modified in the same way as a gradient is modified in error backpropagation in order to enhance the training. Even though these modifications are, in principle, directly applicable, carefully checking and bounding the update steps can improve the performance and can make the procedure more robust. In addition, this paper provides a new and much more detailed experimental evaluation that is based on fuzzy cluster comparison measures [C. Borgelt, Resampling for fuzzy clustering, Int. J. Uncertainty, Fuzziness Knowledge-based Syst. 15 (5) (2007), 595-614], which can be used nicely to study the convergence speed.     

Hierarchical unsupervised fuzzy clustering

A recursive algorithm for hierarchical fuzzy partitioning is presented. The algorithm has the advantages of hierarchical clustering, while maintaining fuzzy clustering rules. Each pattern can have a nonzero membership in more than one subset of the data in the hierarchy. Optimal feature extraction and reduction is optionally reapplied for each subset. Combining hierarchical and fuzzy concepts is suggested as a natural feasible solution to the cluster validity problem of real data. The convergence and membership conservation of the algorithm are proven. The algorithm is shown to be effective for a variety of data sets with a wide dynamic range of both covariance matrices and number of members in each class     

Evolutionary semi-supervised fuzzy clustering

For learning classifier from labeled and unlabeled data, this paper proposes an evolutionary semi-supervised fuzzy clustering algorithm. Class labels information provided by labeled data is used to guide the evolution process of each fuzzy partition on unlabeled data, which plays the role of chromosome. The fitness of each chromosome is evaluated with a combination of fuzzy within cluster variance of unlabeled data and misclassification error of labeled data. The structure of the clusters obtained can be used to classify a future new pattern. The performance of the proposed approach is evaluated using two benchmark data sets. Experimental results indicate that the proposed approach can improve classification accuracy significantly, compared to classifier trained with a small number of labeled data only. Also, it outperforms a similar approach SSFCM.     

Active semi-supervised fuzzy clustering

Clustering algorithms are increasingly employed for the categorization of image databases, in order to provide users with database overviews and make their access more effective. By including information provided by the user, the categorization process can produce results that come closer to user's expectations. To make such a semi-supervised categorization approach acceptable for the user, this information must be of a very simple nature and the amount of information the user is required to provide must be minimized. We propose here an effective semi-supervised clustering algorithm, active fuzzy constrained clustering (AFCC), that minimizes a competitive agglomeration cost function with fuzzy terms corresponding to pairwise constraints provided by the user. In order to minimize the amount of constraints required, we define an active mechanism for the selection of candidate constraints. The comparisons performed on a simple benchmark and on a ground truth image database show that with AFCC the results of clustering can be significantly improved with few constraints, making this semi-supervised approach an attractive alternative in the categorization of image databases.     

基于减法聚类改进的模糊c-均值算法的模糊聚类研究

针对模糊c-均值(FCM)聚类算法受初始聚类中心影响,易陷入局部最优,以及算法对孤立点数据敏感的问题,提出了解决方案:采用快速减法聚类算法初始化聚类中心,为每个样本点赋予一个定量的权值,用来区分不同的样本点对最终的聚类结果的不同作用,为提高聚类速度采用修正隶属度矩阵的方法,并将算法与传统的FCM相比.实验结果表明,该算法较好地解决了初值问题,与随机初始化方法相比,迭代次数少、收敛速度快、具有较好的聚类结果.     

特征加权的模糊C聚类算法

参照文献[5]中将K-means聚类算法与特征权重优化相结合的方法,推导出FCM聚类算法与特征权重优化相结合的优化迭代公式,形成加权FCM算法.将加权FCM算法中计算聚类均值项的公式代入到计算隶属度的更新公式和特征权重的更新公式中,得到加权FCM扩展算法.由于这个扩展算法消去了均值项,它对于有序属性和无序类别属性的隶属度和特征权重的更新公式具有统一的形式,因此可以很方便地应用到混合属性数据集的加权聚类分析中来.该算法的收敛性分析与FCM类似,算法迭代结束后能给出一组优化的特征权重值.仿真实验结果与WKMeans算法的结果基本一致,说明该方法在优化混合属性数据集的特征权重时是有效的.     

Modified fuzzy ants clustering approach

Being trapped in local optima within clustering search space currently is nontrivial difficulty. In order to relieve such a difficulty, even using genetic algorithm to optimize the initial clusters for fuzzy c-means is still unsatisfied. Since genetic algorithm intensifies only the current best solution, it will easily gets trapped in local minima. The ant colony system, dissimilarly to genetic algorithm, recognizes that the solutions near the best solution are also good ones and they bring about smoothness of solution. This paper proposes a modified fuzzy ant clustering. Such a presented method is a combination of genetic algorithm, ant colony system and fuzzy c-means. It is employed in creating fuzzy color histogram in image retrieval application. The performance measurement relates to the percentages of accuracy of image retrieval. Experimental results show that the proposed approach yields the best results among others with respect to sensitivity and robustness on dealing with lighting intensity changes, quantization errors, also changes in number of images and in size of color space, even the certain-range variation of a particular parameter of clustering.     

Spatial dependence-based fuzzy regression clustering

Fuzzy clustering based regression analysis is a novel hybrid approach to capture the linear structure while considering the classification structure of the measurement. Using the concept that weights provided via the fuzzy degree of clustering, some regression models have been proposed in literature. In these models, membership values derived from clustering or some weights obtained from geometrical functions are employed as the weights of regression system. This paper addresses a weighted fuzzy regression analysis based on spatial dependence measure of the memberships. By the methodology presented in this paper, the relative weights are used in fuzzy regression models instead of direct membership values or their geometrical transforms. The experimental studies indicate that the spatial dependence based analyses yield more reliable results to show the correlation of the independent variables into the dependent variable. In addition, it has been observed that spatial dependence based models have high estimation and generalization capacities.     

Generalized weighted conditional fuzzy clustering

Fuzzy clustering helps to find natural vague boundaries in data. The fuzzy c-means method is one of the most popular clustering methods based on minimization of a criterion function. Among many existing modifications of this method, conditional or context-dependent c-means is the most interesting one. In this method, data vectors are clustered under conditions based on linguistic terms represented by fuzzy sets. This paper introduces a family of generalized weighted conditional fuzzy c-means clustering algorithms. This family include both the well-known fuzzy c-means method and the conditional fuzzy c-means method. Performance of the new clustering algorithm is experimentally compared with fuzzy c-means using synthetic data with outliers and the Box-Jenkins database.     

Effective fuzzy c-means clustering algorithms for data clustering problems

Clustering is a well known technique in identifying intrinsic structures and find out useful information from large amount of data. One of the most extensively used clustering techniques is the fuzzy c-means algorithm. However, computational task becomes a problem in standard objective function of fuzzy c-means due to large amount of data, measurement uncertainty in data objects. Further, the fuzzy c-means suffer to set the optimal parameters for the clustering method. Hence the goal of this paper is to produce an alternative generalization of FCM clustering techniques in order to deal with the more complicated data; called quadratic entropy based fuzzy c-means. This paper is dealing with the effective quadratic entropy fuzzy c-means using the combination of regularization function, quadratic terms, mean distance functions, and kernel distance functions. It gives a complete framework of quadratic entropy approaching for constructing effective quadratic entropy based fuzzy clustering algorithms. This paper establishes an effective way of estimating memberships and updating centers by minimizing the proposed objective functions. In order to reduce the number iterations of proposed techniques this article proposes a new algorithm to initialize the cluster centers.In order to obtain the cluster validity and choosing the number of clusters in using proposed techniques, we use silhouette method. First time, this paper segments the synthetic control chart time series directly using our proposed methods for examining the performance of methods and it shows that the proposed clustering techniques have advantages over the existing standard FCM and very recent ClusterM-k-NN in segmenting synthetic control chart time series.     

On tolerant fuzzy c-means clustering and tolerant possibilistic clustering

This paper presents two new types of clustering algorithms by using tolerance vector called tolerant fuzzy c-means clustering and tolerant possibilistic clustering. In the proposed algorithms, the new concept of tolerance vector plays very important role. The original concept is developed to handle data flexibly, that is, a tolerance vector attributes not only to each data but also each cluster. Using the new concept, we can consider the influence of clusters to each data by the tolerance. First, the new concept of tolerance is introduced into optimization problems. Second, the optimization problems with tolerance are solved by using Karush–Kuhn–Tucker conditions. Third, new clustering algorithms are constructed based on the optimal solutions for clustering. Finally, the effectiveness of the proposed algorithms is verified through numerical examples and its fuzzy classification function.     

Granular prototyping in fuzzy clustering

We introduce a logic-driven clustering in which prototypes are formed and evaluated in a sequential manner. The way of revealing a structure in data is realized by maximizing a certain performance index (objective function) that takes into consideration an overall level of matching (to be maximized) and a similarity level between the prototypes (the component to be minimized). The prototypes identified in the process come with the optimal weight vector that serves to indicate the significance of the individual features (coordinates) in the data grouping represented by the prototype. Since the topologies of these groupings are in general quite diverse the optimal weight vectors are reflecting the anisotropy of the feature space, i.e., they show some local ranking of features in the data space. Having found the prototypes we consider an inverse similarity problem and show how the relevance of the prototypes translates into their granularity.     

Intuitionistic fuzzy MST clustering algorithms

In this paper, we investigate graph theory-based clustering techniques for Atanassov’s intuitionistic fuzzy sets (A-IFSs) and interval-valued intuitionistic fuzzy sets (IVIFSs). We start by introducing the concepts of graph, minimum spanning tree (MST), A-IFS, and intuitionistic fuzzy distance, and develop two intuitionistic fuzzy MST clustering algorithms ( and ). Then we extend Algorithm II for clustering IVIFSs, and show the effectiveness of our algorithms through some numerical experiments.     

Fuzzy C-means and fuzzy swarm for fuzzy clustering problem

Fuzzy clustering is an important problem which is the subject of active research in several real-world applications. Fuzzy c-means (FCM) algorithm is one of the most popular fuzzy clustering techniques because it is efficient, straightforward, and easy to implement. However, FCM is sensitive to initialization and is easily trapped in local optima. Particle swarm optimization (PSO) is a stochastic global optimization tool which is used in many optimization problems. In this paper, a hybrid fuzzy clustering method based on FCM and fuzzy PSO (FPSO) is proposed which make use of the merits of both algorithms. Experimental results show that our proposed method is efficient and can reveal encouraging results.     

Multi-variable fuzzy forecasting based on fuzzy clustering and fuzzy rule interpolation techniques

In this paper, we present a new method for multi-variable fuzzy forecasting based on fuzzy clustering and fuzzy rule interpolation techniques. First, the proposed method constructs training samples based on the variation rates of the training data set and then uses the training samples to construct fuzzy rules by making use of the fuzzy C-means clustering algorithm, where each fuzzy rule corresponds to a given cluster. Then, we determine the weight of each fuzzy rule with respect to the input observations and use such weights to determine the predicted output, based on the multiple fuzzy rules interpolation scheme. We apply the proposed method to the temperature prediction problem and the Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) data. The experimental results show that the proposed method produces better forecasting results than several existing methods.     

A validity measure for fuzzy clustering

The authors present a fuzzy validity criterion based on a validity function which identifies compact and separate fuzzy c-partitions without assumptions as to the number of substructures inherent in the data. This function depends on the data set, geometric distance measure, distance between cluster centroids and more importantly on the fuzzy partition generated by any fuzzy algorithm used. The function is mathematically justified via its relationship to a well-defined hard clustering validity function, the separation index for which the condition of uniqueness has already been established. The performance of this validity function compares favorably to that of several others. The application of this validity function to color image segmentation in a computer color vision system for recognition of IC wafer defects which are otherwise impossible to detect using gray-scale image processing is discussed     

Modeling proportional membership in fuzzy clustering

To provide feedback from a cluster structure to the data from which it has been determined, we propose a framework for mining typological structures based on a fuzzy clustering model of how the data are generated from a cluster structure. To relate data entities to cluster prototypes, we assume that the observed entities share parts of the prototypes in such a way that the membership of an entity to a cluster expresses the proportion of the cluster's prototype reflected in the entity (proportional membership). In the generic version of the model, any entity may independently relate to any prototype, which is similar to the assumption underlying the fuzzy c-means criterion. The model is referred to as fuzzy clustering with proportional membership (FCPM). Several versions of the model relaxing the generic assumptions are presented and alternating minimization techniques for them are developed. The results of experimental studies of FCPM versions and the fuzzy c-means algorithm are presented and discussed, especially addressing the issues of fitting the underlying clustering model. An example is given with data in the medical field in which our approach is shown to suit better than more conventional methods.     

A least biased fuzzy clustering method

A new operational definition of cluster is proposed, and a fuzzy clustering algorithm with minimal biases is formulated by making use of the maximum entropy principle to maximize the entropy of the centroids with respect to the data points (clustering entropy). The authors make no assumptions on the number of clusters or their initial positions. For each value of an adimensional scale parameter β', the clustering algorithm makes each data point iterate towards one of the cluster's centroids, so that both hard and fuzzy partitions are obtained. Since the clustering algorithm can make a multiscale analysis of the given data set one can obtain both hierarchy and partitioning type clustering. The relative stability with respect to β' of each cluster structure is defined as the measurement of cluster validity. The authors determine the specific value of β' which corresponds to the optimal positions of cluster centroids by minimizing the entropy of the data points with respect to the centroids (clustered entropy). Examples are given to show how this least biased method succeeds in getting perceptually correct clustering results     

Switching regression models and fuzzy clustering

A family of objective functions called fuzzy c-regression models, which can be used too fit switching regression models to certain types of mixed data, is presented. Minimization of particular objective functions in the family yields simultaneous estimates for the parameters of c regression models, together with a fuzzy c-partitioning of the data. A general optimization approach for the family of objective functions is given and corresponding theoretical convergence results are discussed. The approach is illustrated by two numerical examples that show how it can be used to fit mixed data to coupled linear and nonlinear models     

Spectral clustering with fuzzy similarity measure

Spectral clustering algorithms have been successfully used in the field of pattern recognition and computer vision. The widely used similarity measure for spectral clustering is Gaussian kernel function which measures the similarity between data points. However, it is difficult for spectral clustering to choose the suitable scaling parameter in Gaussian kernel similarity measure. In this paper, utilizing the prototypes and partition matrix obtained by fuzzy c-means clustering algorithm, we develop a fuzzy similarity measure for spectral clustering (FSSC). Furthermore, we introduce the K-nearest neighbor sparse strategy into FSSC and apply the sparse FSSC to texture image segmentation. In our experiments, we firstly perform some experiments on artificial data to verify the efficiency of the proposed fuzzy similarity measure. Then we analyze the parameters sensitivity of our method. Finally, we take self-tuning spectral clustering and Nyström methods for baseline comparisons, and apply these three methods to the synthetic texture and remote sensing image segmentation. The experimental results show that the proposed method is significantly effective and stable.