Suppressed fuzzy c-means clustering algorithm

Based on the defect of rival checked fuzzy c-means clustering algorithm, a new algorithm: suppressed fuzzy c-means clustering algorithm is proposed. The new algorithm overcomes the shortcomings of the original algorithm, establishes more natural and more reasonable relationships between hard c-means clustering algorithm and fuzzy c-means clustering algorithm.     

Intuitionistic fuzzy C-regression by using least squares support vector regression

This paper proposes a novel intuitionistic fuzzy c-least squares support vector regression (IFC-LSSVR) with a Sammon mapping clustering algorithm. Sammon mapping effectively reduces the complexity of raw data, while intuitionistic fuzzy sets (IFSs) can effectively tune the membership of data points, and LSSVR improves the conventional fuzzy c-regression model. The proposed clustering algorithm combines the advantages of IFSs, LSSVR and Sammon mapping for solving actual clustering problems. Moreover, IFC-LSSVR with Sammon mapping adopts particle swarm optimization to obtain optimal parameters. Experiments conducted on a web-based adaptive learning environment and a dataset of wheat varieties demonstrate that the proposed algorithm is more efficient than conventional algorithms, such as the k-means (KM) and fuzzy c-means (FCM) clustering algorithms, in standard measurement indexes. This study thus demonstrates that the proposed model is a credible fuzzy clustering algorithm. The novel method contributes not only to the theoretical aspects of fuzzy clustering, but is also widely applicable in data mining, image systems, rule-based expert systems and prediction problems.     

Fuzzy clustering algorithms based on resolution and their application in image compression

This paper presents an idea of clustering resolution. On the basis of the idea, fuzzy clustering algorithms based on resolution are deduced, which naturally comprise a set of clustering algorithms. Thus, c-means algorithm and fuzzy c-means algorithms are actually special examples in the set. As an application for codebook design in image compression based on vector quantization, fuzzy clustering algorithms based on multiresolution are developed, which are almost prior to conventional algorithms in all aspects.     

Interactive clustering and merging with a new fuzzy expected value

Major problems exist in both crisp and fuzzy clustering algorithms. The fuzzy c-means type of algorithms use weights determined by a power m of inverse distances that remains fixed over all iterations and over all clusters, even though smaller clusters should have a larger m. Our method uses a different “distance” for each cluster that changes over the early iterations to fit the clusters. Comparisons show improved results. We also address other perplexing problems in clustering: (i) find the optimal number K of clusters; (ii) assess the validity of a given clustering; (iii) prevent the selection of seed vectors as initial prototypes from affecting the clustering; (iv) prevent the order of merging from affecting the clustering; and (v) permit the clusters to form more natural shapes rather than forcing them into normed balls of the distance function. We employ a relatively large number K of uniformly randomly distributed seeds and then thin them to leave fewer uniformly distributed seeds. Next, the main loop iterates by assigning the feature vectors and computing new fuzzy prototypes. Our fuzzy merging then merges any clusters that are too close to each other. We use a modified Xie-Bene validity measure as the goodness of clustering measure for multiple values of K in a user-interaction approach where the user selects two parameters (for eliminating clusters and merging clusters after viewing the results thus far). The algorithm is compared with the fuzzy c-means on the iris data and on the Wisconsin breast cancer data.     

Mountain c-regressions method

Since Quandt [The estimation of the parameters of a linear regression system obeying two separate regimes, Journal of the American Statistical Association 53 (1958) 873-880] initiated the research on 2-regressions analysis, switching regression had been widely studied and applied in psychology, economics, social science and music perception. In fuzzy clustering, the fuzzy c-means (FCM) is the most commonly used algorithm. Hathaway and Bezdek [Switching regression models and fuzzy clustering, IEEE Transactions on Fuzzy Systems 1 (1993) 195-204] embedded FCM into switching regression where it was called fuzzy c-regressions (FCR). However, the FCR always depends heavily on initial values. In this paper, we propose a mountain c-regressions (MCR) method for solving the initial-value problem. First, we perform data transformation for the switching regression data set, and then implement the modified mountain clustering on the transformed data to extract c cluster centers. These extracted c cluster centers in the transformed space will correspond to c regression models in the original data set. The proposed MCR method can form well-estimated c regression models for switching regression data sets. According to the properties of transformation, the proposed MCR is also robust to noise and outliers. Several examples show the effectiveness and superiority of our proposed method.     

Unsupervised possibilistic clustering

In fuzzy clustering, the fuzzy c-means (FCM) clustering algorithm is the best known and used method. Since the FCM memberships do not always explain the degrees of belonging for the data well, Krishnapuram and Keller proposed a possibilistic approach to clustering to correct this weakness of FCM. However, the performance of Krishnapuram and Keller's approach depends heavily on the parameters. In this paper, we propose another possibilistic clustering algorithm (PCA) which is based on the FCM objective function, the partition coefficient (PC) and partition entropy (PE) validity indexes. The resulting membership becomes the exponential function, so that it is robust to noise and outliers. The parameters in PCA can be easily handled. Also, the PCA objective function can be considered as a potential function, or a mountain function, so that the prototypes of PCA can be correspondent to the peaks of the estimated function. To validate the clustering results obtained through a PCA, we generalized the validity indexes of FCM. This generalization makes each validity index workable in both fuzzy and possibilistic clustering models. By combining these generalized validity indexes, an unsupervised possibilistic clustering is proposed. Some numerical examples and real data implementation on the basis of the proposed PCA and generalized validity indexes show their effectiveness and accuracy.     

Fuzzy clustering algorithms for unsupervised change detection in remote sensing images

In this paper, we propose a context-sensitive technique for unsupervised change detection in multitemporal remote sensing images. The technique is based on fuzzy clustering approach and takes care of spatial correlation between neighboring pixels of the difference image produced by comparing two images acquired on the same geographical area at different times. Since the ranges of pixel values of the difference image belonging to the two clusters (changed and unchanged) generally have overlap, fuzzy clustering techniques seem to be an appropriate and realistic choice to identify them (as we already know from pattern recognition literatures that fuzzy set can handle this type of situation very well). Two fuzzy clustering algorithms, namely fuzzy c-means (FCM) and Gustafson-Kessel clustering (GKC) algorithms have been used for this task in the proposed work. For clustering purpose various image features are extracted using the neighborhood information of pixels. Hybridization of FCM and GKC with two other optimization techniques, genetic algorithm (GA) and simulated annealing (SA), is made to further enhance the performance. To show the effectiveness of the proposed technique, experiments are conducted on two multispectral and multitemporal remote sensing images. A fuzzy cluster validity index (Xie-Beni) is used to quantitatively evaluate the performance. Results are compared with those of existing Markov random field (MRF) and neural network based algorithms and found to be superior. The proposed technique is less time consuming and unlike MRF does not require any a priori knowledge of distributions of changed and unchanged pixels.     

Spectral clustering with fuzzy similarity measure

Spectral clustering algorithms have been successfully used in the field of pattern recognition and computer vision. The widely used similarity measure for spectral clustering is Gaussian kernel function which measures the similarity between data points. However, it is difficult for spectral clustering to choose the suitable scaling parameter in Gaussian kernel similarity measure. In this paper, utilizing the prototypes and partition matrix obtained by fuzzy c-means clustering algorithm, we develop a fuzzy similarity measure for spectral clustering (FSSC). Furthermore, we introduce the K-nearest neighbor sparse strategy into FSSC and apply the sparse FSSC to texture image segmentation. In our experiments, we firstly perform some experiments on artificial data to verify the efficiency of the proposed fuzzy similarity measure. Then we analyze the parameters sensitivity of our method. Finally, we take self-tuning spectral clustering and Nyström methods for baseline comparisons, and apply these three methods to the synthetic texture and remote sensing image segmentation. The experimental results show that the proposed method is significantly effective and stable.     

Fuzzy wavelet neural network based on fuzzy clustering and gradient techniques for time series prediction

This paper presents the development of fuzzy wavelet neural network system for time series prediction that combines the advantages of fuzzy systems and wavelet neural network. The structure of fuzzy wavelet neural network (FWNN) is proposed, and its learning algorithm is derived. The proposed network is constructed on the base of a set of TSK fuzzy rules that includes a wavelet function in the consequent part of each rule. A fuzzy c-means clustering algorithm is implemented to generate the rules, that is the structure of FWNN prediction model, automatically, and the gradient-learning algorithm is used for parameter identification. The use of fuzzy c-means clustering algorithm with the gradient algorithm allows to improve convergence of learning algorithm. FWNN is used for modeling and prediction of complex time series and prediction of foreign-exchange rates. Exchange rates are dynamic process that changes every day and have high-order nonlinearity. The statistical data for the last 2 years are used for the development of FWNN prediction model. Effectiveness of the proposed system is evaluated with the results obtained from the simulation of FWNN-based systems and with the comparative simulation results of previous related models.     

Sample-weighted clustering methods

Although there have been many researches on cluster analysis considering feature (or variable) weights, little effort has been made regarding sample weights in clustering. In practice, not every sample in a data set has the same importance in cluster analysis. Therefore, it is interesting to obtain the proper sample weights for clustering a data set. In this paper, we consider a probability distribution over a data set to represent its sample weights. We then apply the maximum entropy principle to automatically compute these sample weights for clustering. Such method can generate the sample-weighted versions of most clustering algorithms, such as k-means, fuzzy c-means (FCM) and expectation & maximization (EM), etc. The proposed sample-weighted clustering algorithms will be robust for data sets with noise and outliers. Furthermore, we also analyze the convergence properties of the proposed algorithms. This study also uses some numerical data and real data sets for demonstration and comparison. Experimental results and comparisons actually demonstrate that the proposed sample-weighted clustering algorithms are effective and robust clustering methods.     

Further reduced form of wavelet feature for text independent speaker recognition

A text independent speaker recognition system based on wavelet transform derived from fuzzy c-means clustering is proposed. The fuzzy c-means clustering is applied to the speaker data compression in spectrum domain. A set of experiments are conducted, which gives a 95% recognition rate for 100 Mandarin speakers.     

Improving the performance of k-means for color quantization

Color quantization is an important operation with many applications in graphics and image processing. Most quantization methods are essentially based on data clustering algorithms. However, despite its popularity as a general purpose clustering algorithm, k-means has not received much respect in the color quantization literature because of its high computational requirements and sensitivity to initialization. In this paper, we investigate the performance of k-means as a color quantizer. We implement fast and exact variants of k-means with several initialization schemes and then compare the resulting quantizers to some of the most popular quantizers in the literature. Experiments on a diverse set of images demonstrate that an efficient implementation of k-means with an appropriate initialization strategy can in fact serve as a very effective color quantizer.     

Fast modified global k-means algorithm for incremental cluster construction

The k-means algorithm and its variations are known to be fast clustering algorithms. However, they are sensitive to the choice of starting points and are inefficient for solving clustering problems in large datasets. Recently, incremental approaches have been developed to resolve difficulties with the choice of starting points. The global k-means and the modified global k-means algorithms are based on such an approach. They iteratively add one cluster center at a time. Numerical experiments show that these algorithms considerably improve the k-means algorithm. However, they require storing the whole affinity matrix or computing this matrix at each iteration. This makes both algorithms time consuming and memory demanding for clustering even moderately large datasets. In this paper, a new version of the modified global k-means algorithm is proposed. We introduce an auxiliary cluster function to generate a set of starting points lying in different parts of the dataset. We exploit information gathered in previous iterations of the incremental algorithm to eliminate the need of computing or storing the whole affinity matrix and thereby to reduce computational effort and memory usage. Results of numerical experiments on six standard datasets demonstrate that the new algorithm is more efficient than the global and the modified global k-means algorithms.     

Fast and robust fuzzy c-means clustering algorithms incorporating local information for image segmentation

Fuzzy c-means (FCM) algorithms with spatial constraints (FCM_S) have been proven effective for image segmentation. However, they still have the following disadvantages: (1) although the introduction of local spatial information to the corresponding objective functions enhances their insensitiveness to noise to some extent, they still lack enough robustness to noise and outliers, especially in absence of prior knowledge of the noise; (2) in their objective functions, there exists a crucial parameter α used to balance between robustness to noise and effectiveness of preserving the details of the image, it is selected generally through experience; and (3) the time of segmenting an image is dependent on the image size, and hence the larger the size of the image, the more the segmentation time. In this paper, by incorporating local spatial and gray information together, a novel fast and robust FCM framework for image segmentation, i.e., fast generalized fuzzy c-means (FGFCM) clustering algorithms, is proposed. FGFCM can mitigate the disadvantages of FCM_S and at the same time enhances the clustering performance. Furthermore, FGFCM not only includes many existing algorithms, such as fast FCM and enhanced FCM as its special cases, but also can derive other new algorithms such as FGFCM_S1 and FGFCM_S2 proposed in the rest of this paper. The major characteristics of FGFCM are: (1) to use a new factor S_ij as a local (both spatial and gray) similarity measure aiming to guarantee both noise-immunity and detail-preserving for image, and meanwhile remove the empirically-adjusted parameter α; (2) fast clustering or segmenting image, the segmenting time is only dependent on the number of the gray-levels q rather than the size N(?q) of the image, and consequently its computational complexity is reduced from O(NcI₁) to O(qcI₂), where c is the number of the clusters, I₁ and are the numbers of iterations, respectively, in the standard FCM and our proposed fast segmentation method. The experiments on the synthetic and real-world images show that FGFCM algorithm is effective and efficient.     

Analysis of parameter selections for fuzzy c-means

The weighting exponent m is called the fuzzifier that can influence the performance of fuzzy c-means (FCM). It is generally suggested that m∈[1.5,2.5]. On the basis of a robust analysis of FCM, a new guideline for selecting the parameter m is proposed. We will show that a large m value will make FCM more robust to noise and outliers. However, considerably large m values that are greater than the theoretical upper bound will make the sample mean a unique optimizer. A simple and efficient method to avoid this unexpected case in fuzzy clustering is to assign a cluster core to each cluster. We will also discuss some clustering algorithms that extend FCM to contain the cluster cores in fuzzy clusters. For a large theoretical upper bound case, we suggest the implementation of the FCM with a suitable large m value. Otherwise, we suggest implementing the clustering methods with cluster cores. When the data set contains noise and outliers, the fuzzifier m=4 is recommended for both FCM and cluster-core-based methods in a large theoretical upper bound case.     

Robust fuzzy clustering neural network based on ɛ-insensitive loss function

In the paper, as an improvement of fuzzy clustering neural network FCNN proposed by Zhang et al., a novel robust fuzzy clustering neural network RFCNN is presented to cope with the sensitive issue of clustering when outliers exist. This new algorithm is based on Vapnik's ɛ-insensitive loss function and quadratic programming optimization. Our experimental results demonstrate that RFCNN has much better robustness for outliers than FCNN.     

A hybrid genetic algorithm–fuzzy c-means approach for incomplete data clustering based on nearest-neighbor intervals

Incomplete data are often encountered in data sets used in clustering problems, and inappropriate treatment of incomplete data can significantly degrade the clustering performance. In view of the uncertainty of missing attributes, we put forward an interval representation of missing attributes based on nearest-neighbor information, named nearest-neighbor interval, and a hybrid approach utilizing genetic algorithm and fuzzy c-means is presented for incomplete data clustering. The overall algorithm is within the genetic algorithm framework, which searches for appropriate imputations of missing attributes in corresponding nearest-neighbor intervals to recover the incomplete data set, and hybridizes fuzzy c-means to perform clustering analysis and provide fitness metric for genetic optimization simultaneously. Several experimental results on a set of real-life data sets are presented to demonstrate the better clustering performance of our hybrid approach over the compared methods.     

On semi-supervised fuzzy c-means clustering for data with clusterwise tolerance by opposite criteria

This paper presents a new semi-supervised fuzzy c-means clustering for data with clusterwise tolerance by opposite criteria. In semi-supervised clustering, pairwise constraints, that is, must-link and cannot-link, are frequently used in order to improve clustering performances. From the viewpoint of handling pairwise constraints, a new semi-supervised fuzzy c-means clustering is proposed by introducing clusterwise tolerance-based pairwise constraints. First, a concept of clusterwise tolerance-based pairwise constraints is introduced. Second, the optimization problems of the proposed method are formulated. Especially, must-link and cannot-link are handled by opposite criteria in our proposed method. Third, a new clustering algorithm is constructed based on the above discussions. Finally, the effectiveness of the proposed algorithm is verified through numerical examples.     

A hierarchical evolutionary algorithm for automatic medical image segmentation

Image segmentation denotes a process of partitioning an image into distinct regions. A large variety of different segmentation approaches for images have been developed. Among them, the clustering methods have been extensively investigated and used. In this paper, a clustering based approach using a hierarchical evolutionary algorithm (HEA) is proposed for medical image segmentation. The HEA can be viewed as a variant of conventional genetic algorithms. By means of a hierarchical structure in the chromosome, the proposed approach can automatically classify the image into appropriate classes and avoid the difficulty of searching for the proper number of classes. The experimental results indicate that the proposed approach can produce more continuous and smoother segmentation results in comparison with four existing methods, competitive Hopfield neural networks (CHNN), dynamic thresholding, k-means, and fuzzy c-means methods.