Optimality test for generalized FCM and its application to parameter selection

In cluster analysis, the fuzzy c-means (FCM) clustering algorithm is the best known and most widely used method. It was proven to converge to either a local minimum or saddle points by Bezdek et al. Wei and Mendel produced efficient optimality tests for FCM fixed points. Recently, a weighting exponent selection for FCM was proposed by Yu et al. Inspired by these results, we unify several alternative FCM algorithms into one model, called the generalized fuzzy c-means (GFCM). This GFCM model presents a wide variation of FCM algorithms and can easily lead to new and interesting clustering algorithms. Moreover, we construct a general optimality test for GFCM fixed points. This is applied to theoretically choose the parameters in the GFCM model. The experimental results demonstrate the precision of the theoretical analysis.     

Sample-weighted clustering methods

Although there have been many researches on cluster analysis considering feature (or variable) weights, little effort has been made regarding sample weights in clustering. In practice, not every sample in a data set has the same importance in cluster analysis. Therefore, it is interesting to obtain the proper sample weights for clustering a data set. In this paper, we consider a probability distribution over a data set to represent its sample weights. We then apply the maximum entropy principle to automatically compute these sample weights for clustering. Such method can generate the sample-weighted versions of most clustering algorithms, such as k-means, fuzzy c-means (FCM) and expectation & maximization (EM), etc. The proposed sample-weighted clustering algorithms will be robust for data sets with noise and outliers. Furthermore, we also analyze the convergence properties of the proposed algorithms. This study also uses some numerical data and real data sets for demonstration and comparison. Experimental results and comparisons actually demonstrate that the proposed sample-weighted clustering algorithms are effective and robust clustering methods.     

Double indices-induced FCM clustering and its integration with fuzzy subspace clustering

As one of the most popular algorithms for cluster analysis, fuzzy c-means (FCM) and its variants have been widely studied. In this paper, a novel generalized version called double indices-induced FCM (DI-FCM) is developed from another perspective. DI-FCM introduces a power exponent r into the constraints of the objective function such that the fuzziness index m is generalized and a new criterion of selecting an appropriate fuzziness index m is defined. Furthermore, it can be explained from the viewpoint of entropy concept that the power exponent r facilitates the introduction of entropy-based constraints into fuzzy clustering algorithms. As an attractive and judicious application, DI-FCM is integrated with a fuzzy subspace clustering (FSC) algorithm so that a new fuzzy subspace clustering algorithm called double indices-induced fuzzy subspace clustering (DI-FSC) algorithm is proposed for high-dimensional data. DI-FSC replaces the commonly used Euclidean distance with the feature-weighted distance, resulting in having two fuzzy matrices in the objective function. A convergence proof of DI-FSC is also established by applying Zangwill’s convergence theorem. Several experiments on both artificial data and real data were conducted and the experimental results show the effectiveness of the proposed algorithm.     

熵指数约束的模糊聚类新算法

针对基于模糊C均值聚类(fuzzy C-means, FCM)算法框架的竞争聚集聚类(competitive agglomeration, CA)算法中模糊指数m被限定为2的问题,提出了一种更为普适的模糊聚类新算法.该算法首先在FCM算法框架的基础上引入熵指数约束条件,构造了基于熵指数约束的模糊C均值聚类(entropy index constraint FCM, EIC-FCM)算法,成功地将模糊指数m>1的约束条件转换为熵指数0相似文献   

Unsupervised possibilistic clustering

In fuzzy clustering, the fuzzy c-means (FCM) clustering algorithm is the best known and used method. Since the FCM memberships do not always explain the degrees of belonging for the data well, Krishnapuram and Keller proposed a possibilistic approach to clustering to correct this weakness of FCM. However, the performance of Krishnapuram and Keller's approach depends heavily on the parameters. In this paper, we propose another possibilistic clustering algorithm (PCA) which is based on the FCM objective function, the partition coefficient (PC) and partition entropy (PE) validity indexes. The resulting membership becomes the exponential function, so that it is robust to noise and outliers. The parameters in PCA can be easily handled. Also, the PCA objective function can be considered as a potential function, or a mountain function, so that the prototypes of PCA can be correspondent to the peaks of the estimated function. To validate the clustering results obtained through a PCA, we generalized the validity indexes of FCM. This generalization makes each validity index workable in both fuzzy and possibilistic clustering models. By combining these generalized validity indexes, an unsupervised possibilistic clustering is proposed. Some numerical examples and real data implementation on the basis of the proposed PCA and generalized validity indexes show their effectiveness and accuracy.     

Intuitionistic fuzzy C-regression by using least squares support vector regression

This paper proposes a novel intuitionistic fuzzy c-least squares support vector regression (IFC-LSSVR) with a Sammon mapping clustering algorithm. Sammon mapping effectively reduces the complexity of raw data, while intuitionistic fuzzy sets (IFSs) can effectively tune the membership of data points, and LSSVR improves the conventional fuzzy c-regression model. The proposed clustering algorithm combines the advantages of IFSs, LSSVR and Sammon mapping for solving actual clustering problems. Moreover, IFC-LSSVR with Sammon mapping adopts particle swarm optimization to obtain optimal parameters. Experiments conducted on a web-based adaptive learning environment and a dataset of wheat varieties demonstrate that the proposed algorithm is more efficient than conventional algorithms, such as the k-means (KM) and fuzzy c-means (FCM) clustering algorithms, in standard measurement indexes. This study thus demonstrates that the proposed model is a credible fuzzy clustering algorithm. The novel method contributes not only to the theoretical aspects of fuzzy clustering, but is also widely applicable in data mining, image systems, rule-based expert systems and prediction problems.     

Analytical and numerical evaluation of the suppressed fuzzy c-means algorithm: a study on the competition in c-means clustering models

Suppressed fuzzy c-means (s-FCM) clustering was introduced in Fan et al. (Pattern Recogn Lett 24:1607–1612, 2003) with the intention of combining the higher speed of hard c-means (HCM) clustering with the better classification properties of fuzzy c-means (FCM) algorithm. The authors modified the FCM iteration to create a competition among clusters: lower degrees of memberships were diminished according to a previously set suppression rate, while the largest fuzzy membership grew by swallowing all the suppressed parts of the small ones. Suppressing the FCM algorithm was found successful in the terms of accuracy and working time, but the authors failed to answer a series of important questions. In this paper, we clarify the view upon the optimality and the competitive behavior of s-FCM via analytical computations and numerical analysis. A quasi competitive learning rate (QLR) is introduced first, in order to quantify the effect of suppression. As the investigation of s-FCM’s optimality did not provide a precise result, an alternative, optimally suppressed FCM (Os-FCM) algorithm is proposed as a hybridization of FCM and HCM. Both the suppressed and optimally suppressed FCM algorithms underwent the same analytical and numerical evaluations, their properties were analyzed using the QLR. We found the newly introduced Os-FCM algorithm quicker than s-FCM at any nontrivial suppression level. Os-FCM should also be favored because of its guaranteed optimality.     

直觉模糊C-均值聚类算法研究

鉴于直觉模糊集理论作为模糊理论的推广已得到广泛的应用,研究了将模糊C-均值聚类推广为直觉模糊C-均值聚类（IFCM）的途径和方法,分析了现有的几种IFCM算法,并提出了一种基于直觉模糊集的模糊C-均值聚类算法．该算法首先定义了直觉模糊集之间的距离;然后构造了聚类的目标函数;最后给出了聚类算法步骤．将算法用于目标识别,实验结果表明了算法的有效性．     

标签分布熵正则的模糊C均值平衡聚类方法

许多应用场景要求每个类别的数量相对平衡,而传统模糊C均值(FCM)聚类算法无法实现此功能.为此,利用标签信息构造标签分布熵评价聚类的平衡度,然后将标签分布熵、模糊隶属度矩阵与标签矩阵之间的平方损失同时引入到传统FCM中,进而提出一种标签分布熵正则的模糊C均值平衡聚类方法 (FCMLDE).同时,利用迭代方法和增广拉格朗日乘数法设计该模型的优化算法.最后,利用6个真实数据集进行聚类实验,结果表明,所提方法在聚类性能和平衡性能上均具有很好的优势.     

Convergence properties of the generalized fuzzy c-means clustering algorithms

Fuzzy c-means (FCM) clustering algorithms have been widely used to solve clustering problems. Yang and Yu [1] extended these to optimization procedures with respect to any probability distribution. They showed that the optimal cluster centers are the fixed points of these generalized FCM clustering algorithms. The convergence properties of algorithms are the important theoretical issue. In this paper, we present convergence properties of the generalized FCM clustering algorithms. These are global convergence, local convergence, and its rate of convergence.     

一种基于决策粗糙集的模糊C均值聚类数的确定方法

Fuzzy C-Means(FCM)是模糊聚类中聚类效果较好且应用较为广泛的聚类算法,但是其对初始聚类数的敏感性导致如何选择一个较好的C值 变得十分重要。因此,确定FCM的聚类数是使用FCM进行聚类分析时的一个至关重要的步骤。通过扩展决策粗糙集模型进行聚类的有效性分析,并进一步确定FCM的聚类数,从而避免了使用FCM时不好的初始化所带来的影响。文中提出了一种基于扩展粗糙集模型的模糊C均值聚类数的确定方法,并通过图像分割实验来验证聚类的效果。实验通过比对不同聚类数下分类结果的代价获得了一个较好的分割结果,并将结果与Z.Yu等人于2015年提出的蚁群模糊C均值混合算法(AFHA)以及提高的AFHA算法(IAFHA)进行对比,结果表明所提方法的聚类结果较好,图像分割效果较明显,Bezdek分割系数比AFHA和IAFHA算法的更高,且在Xie-Beni系数上也有较大优势。     

A contribution to convergence theory of fuzzy c-means and derivatives

In this paper, we revisit the convergence and optimization properties of fuzzy clustering algorithms, in general, and the fuzzy c-means (FCM) algorithm, in particular. Our investigation includes probabilistic and (a slightly modified implementation of) possibilistic memberships, which will be discussed under a unified view. We give a convergence proof for the axis-parallel variant of the algorithm by Gustafson and Kessel, that can be generalized to other algorithms more easily than in the usual approach. Using reformulated fuzzy clustering algorithms, we apply Banach's classical contraction principle and establish a relationship between saddle points and attractive fixed points. For the special case of FCM we derive a sufficient condition for fixed points to be attractive, allowing identification of them as (local) minima of the objective function (excluding the possibility of a saddle point).     

Effective fuzzy c-means clustering algorithms for data clustering problems

Clustering is a well known technique in identifying intrinsic structures and find out useful information from large amount of data. One of the most extensively used clustering techniques is the fuzzy c-means algorithm. However, computational task becomes a problem in standard objective function of fuzzy c-means due to large amount of data, measurement uncertainty in data objects. Further, the fuzzy c-means suffer to set the optimal parameters for the clustering method. Hence the goal of this paper is to produce an alternative generalization of FCM clustering techniques in order to deal with the more complicated data; called quadratic entropy based fuzzy c-means. This paper is dealing with the effective quadratic entropy fuzzy c-means using the combination of regularization function, quadratic terms, mean distance functions, and kernel distance functions. It gives a complete framework of quadratic entropy approaching for constructing effective quadratic entropy based fuzzy clustering algorithms. This paper establishes an effective way of estimating memberships and updating centers by minimizing the proposed objective functions. In order to reduce the number iterations of proposed techniques this article proposes a new algorithm to initialize the cluster centers.In order to obtain the cluster validity and choosing the number of clusters in using proposed techniques, we use silhouette method. First time, this paper segments the synthetic control chart time series directly using our proposed methods for examining the performance of methods and it shows that the proposed clustering techniques have advantages over the existing standard FCM and very recent ClusterM-k-NN in segmenting synthetic control chart time series.     

Alpha-cut implemented fuzzy clustering algorithms and switching regressions.

In the fuzzy c-means (FCM) clustering algorithm, almost none of the data points have a membership value of 1. Moreover, noise and outliers may cause difficulties in obtaining appropriate clustering results from the FCM algorithm. The embedding of FCM into switching regressions, called the fuzzy c-regressions (FCRs), still has the same drawbacks as FCM. In this paper, we propose the alpha-cut implemented fuzzy clustering algorithms, referred to as FCMalpha, which allow the data points being able to completely belong to one cluster. The proposed FCMalpha algorithms can form a cluster core for each cluster, where data points inside a cluster core will have a membership value of 1 so that it can resolve the drawbacks of FCM. On the other hand, the fuzziness index m plays different roles for FCM and FCMalpha. We find that the clustering results obtained by FCMalpha are more robust to noise and outliers than FCM when a larger m is used. Moreover, the cluster cores generated by FCMalpha are workable for various data shape clusters, so that FCMalpha is very suitable for embedding into switching regressions. The embedding of FCMalpha into switching regressions is called FCRalpha. The proposed FCRalpha provides better results than FCR for environments with noise or outliers. Numerical examples show the robustness and the superiority of our proposed methods.     

Regularized Linear Fuzzy Clustering and Probabilistic PCA Mixture Models

Fuzzy$c$-means (FCM)-type fuzzy clustering approaches are closely related to Gaussian mixture models (GMMs) and EM-like algorithms have been used in FCM clustering with regularized objective functions. Especially, FCM with regularization by Kullback–Leibler information (KLFCM) is a fuzzy counterpart of GMMs. In this paper, we propose to apply probabilistic principal component analysis (PCA) mixture models to linear clustering following a discussion on the relationship between local PCA and linear fuzzy clustering. Although the proposed method is a kind of the constrained model of KLFCM, the algorithm includes the fuzzy$c$-varieties (FCV) algorithm as a special case, and the algorithm can be regarded as a modified FCV algorithm with regularization by K–L information. Numerical experiments demonstrate that the proposed clustering algorithm is more flexible than the maximum likelihood approaches and is useful for capturing local substructures properly.     

Fuzzy and possibilistic shell clustering algorithms and theirapplication to boundary detection and surface approximation. I

Traditionally, prototype-based fuzzy clustering algorithms such as the Fuzzy C Means (FCM) algorithm have been used to find “compact” or “filled” clusters. Recently, there have been attempts to generalize such algorithms to the case of hollow or “shell-like” clusters, i.e., clusters that lie in subspaces of feature space. The shell clustering approach provides a powerful means to solve the hitherto unsolved problem of simultaneously fitting multiple curves/surfaces to unsegmented, scattered and sparse data. In this paper, we present several fuzzy and possibilistic algorithms to detect linear and quadric shell clusters. We also introduce generalizations of these algorithms in which the prototypes represent sets of higher-order polynomial functions. The suggested algorithms provide a good trade-off between computational complexity and performance, since the objective function used in these algorithms is the sum of squared distances, and the clustering is sensitive to noise and outliers. We show that by using a possibilistic approach to clustering, one can make the proposed algorithms robust     

Analysis of parameter selections for fuzzy c-means

The weighting exponent m is called the fuzzifier that can influence the performance of fuzzy c-means (FCM). It is generally suggested that m∈[1.5,2.5]. On the basis of a robust analysis of FCM, a new guideline for selecting the parameter m is proposed. We will show that a large m value will make FCM more robust to noise and outliers. However, considerably large m values that are greater than the theoretical upper bound will make the sample mean a unique optimizer. A simple and efficient method to avoid this unexpected case in fuzzy clustering is to assign a cluster core to each cluster. We will also discuss some clustering algorithms that extend FCM to contain the cluster cores in fuzzy clusters. For a large theoretical upper bound case, we suggest the implementation of the FCM with a suitable large m value. Otherwise, we suggest implementing the clustering methods with cluster cores. When the data set contains noise and outliers, the fuzzifier m=4 is recommended for both FCM and cluster-core-based methods in a large theoretical upper bound case.     

基于知识利用的迁移学习一般化增强模糊划分聚类算法

针对非充分数据集及噪声对聚类分析的干扰,基于模糊C均值(FCM)框架下的聚类技术,即一般化的增强模糊划分聚类算法(GIFP-FCM),探讨具有迁移学习能力的聚类方法--融入迁移学习机制的GIFP-FCM算法(T-GIFP-FCM)。该算法通过有效利用历史相关场景(域)总结得到的知识来指导当前场景(域)中信息不足时的聚类任务,从而提高聚类效果。通过在模拟数据集及真实数据集上的仿真实验,结果显示文中算法较之传统算法在处理信息不足任务时具有更佳的性能。     

FCM-Based Model Selection Algorithms for Determining the Number of Clusters

Clustering is an important research topic that has practical applications in many fields. It has been demonstrated that fuzzy clustering, using algorithms such as the fuzzy C-means (FCM), has clear advantages over crisp and probabilistic clustering methods. Like most clustering algorithms, however, FCM and its derivatives need the number of clusters in the given data set as one of their initializing parameters. The main goal of this paper is to develop an effective fuzzy algorithm for automatically determining the number of clusters. After a brief review of the relevant literature, we present a new algorithm for determining the number of clusters in a given data set and a new validity index for measuring the “goodness” of clustering. Experimental results and comparisons are given to illustrate the performance of the new algorithm.     

Factor Analysis Latent Subspace Modeling and Robust Fuzzy Clustering Using $t$-Distributions

Factor analysis is a latent subspace model commonly used for local dimensionality reduction tasks. Fuzzy $c$-means (FCM) type fuzzy clustering approaches are closely related to Gaussian mixture models (GMMs), and expectation--maximization (EM) like algorithms have been employed in fuzzy clustering with regularized objective functions. Student's $t$ -mixture models (SMMs) have been proposed recently as an alternative to GMMs, resolving their outlier vulnerability problems. In this paper, we propose a novel FCM-type fuzzy clustering scheme providing two significant benefits when compared with the existing approaches. First, it provides a well-established observation space dimensionality reduction framework for fuzzy clustering algorithms based on factor analysis, allowing concurrent performance of fuzzy clustering and, within each cluster, local dimensionality reduction. Second, it exploits the outlier tolerance advantages of SMMs to provide a novel, soundly founded, nonheuristic, robust fuzzy clustering framework by introducing the effective means to incorporate the explicit assumption about Student's $t$ -distributed data into the fuzzy clustering procedure. This way, the proposed model yields a significant performance increase for the fuzzy clustering algorithm, as we experimentally demonstrate.