Effective fuzzy c-means clustering algorithms for data clustering problems

Clustering is a well known technique in identifying intrinsic structures and find out useful information from large amount of data. One of the most extensively used clustering techniques is the fuzzy c-means algorithm. However, computational task becomes a problem in standard objective function of fuzzy c-means due to large amount of data, measurement uncertainty in data objects. Further, the fuzzy c-means suffer to set the optimal parameters for the clustering method. Hence the goal of this paper is to produce an alternative generalization of FCM clustering techniques in order to deal with the more complicated data; called quadratic entropy based fuzzy c-means. This paper is dealing with the effective quadratic entropy fuzzy c-means using the combination of regularization function, quadratic terms, mean distance functions, and kernel distance functions. It gives a complete framework of quadratic entropy approaching for constructing effective quadratic entropy based fuzzy clustering algorithms. This paper establishes an effective way of estimating memberships and updating centers by minimizing the proposed objective functions. In order to reduce the number iterations of proposed techniques this article proposes a new algorithm to initialize the cluster centers.In order to obtain the cluster validity and choosing the number of clusters in using proposed techniques, we use silhouette method. First time, this paper segments the synthetic control chart time series directly using our proposed methods for examining the performance of methods and it shows that the proposed clustering techniques have advantages over the existing standard FCM and very recent ClusterM-k-NN in segmenting synthetic control chart time series.     

Supervised fuzzy clustering for rule extraction

This paper deals with the application of orthogonal transforms and fuzzy clustering to extract fuzzy rules from data. It is proposed to use the orthogonal least squares method to supervise the progress of the fuzzy clustering algorithm and remove clusters of less importance with respect to describing the data. Clustering takes place in the product space of systems inputs and outputs and each cluster corresponds to a fuzzy IF-THEN rule. By initializing the clustering with an overestimated number of clusters and subsequently remove less important ones as the clustering progresses, it is sought to obtain a suitable partition of the data in an automated manner. The approach is generally applicable to the fuzzy c-means and related algorithms. The adaptive distance norm fuzzy clustering is studied and applied to the identification of Takagi-Sugeno type rules. Both a synthetic example as well as a real-world modeling problem are considered to illustrate the working and the applicability of the algorithm     

Extended Gaussian kernel version of fuzzy c-means in the problem of data analyzing

Fuzzy c-means clustering with spatial constraints is considered as suitable algorithm for data clustering or data analyzing. But FCM has still lacks enough robustness to employ with noise data, because of its Euclidean distance measure objective function for finding the relationship between the objects. It can only be effective in clustering ‘spherical’ clusters, and it may not give reasonable clustering results for “non-compactly filled” spherical data such as “annular-shaped” data. This paper realized the drawbacks of the general fuzzy c-mean algorithm and it tries to introduce an extended Gaussian version of fuzzy C-means by replacing the Euclidean distance in the original object function of FCM. Firstly, this paper proposes initial kernel version of fuzzy c-means to aim at simplifying its computation and then extended it to extended Gaussian kernel version of fuzzy c-means. It derives an effective method to construct the membership matrix for objects, and it derives a robust method for updating centers from extended Gaussian version of fuzzy C-means. Furthermore, this paper proposes a new prototypes learning method and it obtains initial cluster centers using new mathematical initialization centers for the new effective objective function of fuzzy c-means, so that this paper tries to minimize the iteration of algorithms to obtain more accurate result. Initial experiment will be done with an artificially generated data to show how effectively the new proposed Gaussian version of fuzzy C-means works in obtaining clusters, and then the proposed methods can be implemented to cluster the Wisconsin breast cancer database into two clusters for the classes benign and malignant. To show the effective performance of proposed fuzzy c-means with new initialization of centers of clusters, this work compares the results with results of recent fuzzy c-means algorithm; in addition, it uses Silhouette method to validate the obtained clusters from breast cancer datasets.     

一种基于Mahalanobis距离的增量聚类算法

经典的模糊c均值聚类算法对非球型或椭球型分布的数据集进行聚类效果较差。将经典的模糊c均值聚类中的欧氏距离用Mahalanobis距离替代,利用Mahalanobis距离的优点,将其用于增量学习中,提出一种基于马氏距离的模糊增量聚类学习算法。实验结果表明该算法能较有效地解决模糊聚类方法中的缺陷,提高了训练精度。     

Fuzzy c-means improvement using relaxed constraints support vector machines

Fuzzy clustering is a widely applied method for extracting the underlying models within data. It has been applied successfully in many real-world applications. Fuzzy c-means is one of the most popular fuzzy clustering methods because it produces reasonable results and its implementation is straightforward. One problem with all fuzzy clustering algorithms such as fuzzy c-means is that some data points which are assigned to some clusters have low membership values. It is possible that many samples may be assigned to a cluster with low-confidence. In this paper, an efficient and noise-aware implementation of support vector machines, namely relaxed constraints support vector machines, is used to solve the mentioned problem and improve the performance of fuzzy c-means algorithm. First, fuzzy c-means partitions data into appropriate clusters. Then, the samples with high membership values in each cluster are selected for training a multi-class relaxed constraints support vector machine classifier. Finally, the class labels of the remaining data points are predicted by the latter classifier. The performance of the proposed clustering method is evaluated by quantitative measures such as cluster entropy and Minkowski scores. Experimental results on real-life data sets show the superiority of the proposed method.     

Double indices-induced FCM clustering and its integration with fuzzy subspace clustering

As one of the most popular algorithms for cluster analysis, fuzzy c-means (FCM) and its variants have been widely studied. In this paper, a novel generalized version called double indices-induced FCM (DI-FCM) is developed from another perspective. DI-FCM introduces a power exponent r into the constraints of the objective function such that the fuzziness index m is generalized and a new criterion of selecting an appropriate fuzziness index m is defined. Furthermore, it can be explained from the viewpoint of entropy concept that the power exponent r facilitates the introduction of entropy-based constraints into fuzzy clustering algorithms. As an attractive and judicious application, DI-FCM is integrated with a fuzzy subspace clustering (FSC) algorithm so that a new fuzzy subspace clustering algorithm called double indices-induced fuzzy subspace clustering (DI-FSC) algorithm is proposed for high-dimensional data. DI-FSC replaces the commonly used Euclidean distance with the feature-weighted distance, resulting in having two fuzzy matrices in the objective function. A convergence proof of DI-FSC is also established by applying Zangwill’s convergence theorem. Several experiments on both artificial data and real data were conducted and the experimental results show the effectiveness of the proposed algorithm.     

Image Segmentation Based on Adaptive Cluster Prototype Estimation

An image segmentation algorithm based on adaptive fuzzy c-means (FCM) clustering is presented in this paper. In the conventional FCM clustering algorithm, cluster assignment is based solely on the distribution of pixel attributes in the feature space, and does not take into consideration the spatial distribution of pixels in an image. By introducing a novel dissimilarity index in the modified FCM objective function, the new adaptive fuzzy clustering algorithm is capable of utilizing local contextual information to impose local spatial continuity, thus exploiting the high inter-pixel correlation inherent in most real-world images. The incorporation of local spatial continuity allows the suppression of noise and helps to resolve classification ambiguity. To account for smooth intensity variation within each homogenous region in an image, a multiplicative field is introduced to each of the fixed FCM cluster prototype. The multiplicative field effectively makes the fixed cluster prototype adaptive to slow smooth within-cluster intensity variation, and allows homogenous regions with slow smooth intensity variation to be segmented as a whole. Experimental results with synthetic and real color images have shown the effectiveness of the proposed algorithm.     

一种隶属关系不确定的可能性模糊聚类方法

模糊聚类是聚类分析的一个重要分支,模糊C-均值聚类算法及其改进算法都是一种基于概率约束的聚类方法,所采用隶属度的取值形式体现了数据集的绝对隶属程度,常常出现不理想的聚类结果.对此,提出了不确定隶属的概念,在此基础上,通过提出两个基于相对隶属程度的判断准则参数,设计出一种新的基于隶属关系不确定的可能性模糊聚类新算法,并给出了具体算法实现.新算法将迭代过程中数据集对聚类簇隶属的可能性与不确定性关系引入目标函数中,达到明显的优化聚类结果的功效.理论分析和实验结果表明,相对其他聚类算法,新算法具有更高的聚类正确率.     

基于加权模糊C均值聚类的空预器积灰监测研究

电站空预器积灰会严重影响机组运行经济性.提出加权模糊C均值聚类算法对空预器积灰程度进行监测,该方法计算多维样本中每一维数据的标准差,将其作为权重,计算样本与类心之间的加权欧式距离,降低模糊C均值聚类算法对离群点的敏感度.利用人工数据对该方法进行验证,结果表明,相比于传统模糊C均值聚类算法,提出的方法对离群点识别更加准确...     

Kernel generalized fuzzy c-means clustering with spatial information for image segmentation

The generalized fuzzy c-means clustering algorithm with improved fuzzy partition (GFCM) is a novel modified version of the fuzzy c-means clustering algorithm (FCM). GFCM under appropriate parameters can converge more rapidly than FCM. However, it is found that GFCM is sensitive to noise in gray images. In order to overcome GFCM?s sensitivity to noise in the image, a kernel version of GFCM with spatial information is proposed. In this method, first a term about the spatial constraints derived from the image is introduced into the objective function of GFCM, and then the kernel induced distance is adopted to substitute the Euclidean distance in the new objective function. Experimental results show that the proposed method behaves well in segmentation performance and convergence speed for gray images corrupted by noise.     

Fuzzy clustering with volume prototypes and adaptive cluster merging

Two extensions to objective function-based fuzzy clustering are proposed. First, the (point) prototypes are extended to hypervolumes, whose size can be fixed or can be determined automatically from the data being clustered. It is shown that clustering with hypervolume prototypes can be formulated as the minimization of an objective function. Second, a heuristic cluster merging step is introduced where the similarity among the clusters is assessed during optimization. Starting with an overestimation of the number of clusters in the data, similar clusters are merged in order to obtain a suitable partitioning. An adaptive threshold for merging is proposed. The extensions proposed are applied to Gustafson-Kessel and fuzzy c-means algorithms, and the resulting extended algorithm is given. The properties of the new algorithm are illustrated by various examples.     

Parameter identification of PWARX models using fuzzy distance weighted least squares method

PieceWise AutoRegressive eXogenous (PWARX) models represent one of the broad classes of the hybrid dynamical systems (HDS). Among many classes of HDS, PWARX model used as an attractive modeling structure due to its equivalence to other classes. This paper presents a novel fuzzy distance weight matrix based parameter identification method for PWARX model. In the first phase of the proposed method estimation for the number of affine submodels present in the HDS is proposed using fuzzy clustering validation based algorithm. For the given set of input–output data points generated by predefined PWARX model fuzzy c-means (FCM) clustering procedure is used to classify the data set according to its affine submodels. The fuzzy distance weight matrix based weighted least squares (WLS) algorithm is proposed to identify the parameters for each PWARX submodel, which minimizes the effect of noise and classification error. In the final phase, fuzzy validity function based model selection method is applied to validate the identified PWARX model. The effectiveness of the proposed method is demonstrated using three benchmark examples. Simulation experiments show validation of the proposed method.     

Automated clustering of trajectory data using a particle swarm optimization

Clustering trajectory data discovers and visualizes available structure in movement patterns of mobile objects and has numerous potential applications in traffic control, urban planning, astronomy, and animal science. In this paper, an automated technique for clustering trajectory data using a Particle Swarm Optimization (PSO) approach has been proposed, and Dynamic Time Warping (DTW) distance as one of the most commonly-used distance measures for trajectory data is considered. The proposed technique is able to find (near) optimal number of clusters as well as (near) optimal cluster centers during the clustering process. To reduce the dimensionality of the search space and improve the performance of the proposed method (in terms of a certain performance index), a Discrete Cosine Transform (DCT) representation of cluster centers is considered. The proposed method is able to admit various cluster validity indexes as objective function for optimization. Experimental results over both synthetic and real-world datasets indicate the superiority of the proposed technique to fuzzy C-means, fuzzy K-medoids, and two evolutionary-based clustering techniques proposed in the literature.     

一种基于特征属性的Web用户模糊聚类改进算法

为降低传统FCM算法的计算复杂性,提高Web用户聚类的效果,文中提出了一种改进的基于特征属性的Web用户模糊聚类算法。首先通过用户访问页面的次数和时间建立Web用户兴趣度矩阵,并根据商品的特征属性值将Web用户兴趣度矩阵映射为用户对特征属性的偏好矩阵,从而有效降低数据稀疏性;然后以此为数据集,对传统的FCM算法进行了改进,将聚类中心分为活动和稳定两种,忽略稳定聚类中的距离计算以降低计算复杂性。最后通过仿真实验证实了新算法的有效性和可行性。     

基于优化分块颜色直方图及模糊C聚类的彩色图像检索方法

将数据挖掘的聚类算法应用到基于内容的图像检索中可以有效提高检索的速度和效果。模糊聚类算法更符合图像检索本身所具有的模糊性,但这种方法存在聚类分析时间过久影响检索性能的问题,因此本文提出了一种基于优化分块颜色直方图及模糊C聚类的彩色图像检索方法。首先对图像库中的每幅图像进行分块,并提取出每一块的优化颜色特征信息;然后采用模糊C均值聚类算法对得到的颜色特征向量进行聚类,得到每个图像类的聚类中心;最后计算查询示例图像和对应图像类的图像之间的相似度,按照相似度的大小返回检索结果。实验表明,本文提出的方法不仅具有较高的查全率和查准率,而且提取的特征维数较少,聚类时间短,检索速度快。     

ADAPTIVE HYPER-FUZZY PARTITION PARTICLE SWARM OPTIMIZATION CLUSTERING ALGORITHM

This article presents an adaptive hyper-fuzzy partition particle swarm optimization clustering algorithm to optimally classify different geometrical structure data sets into correct groups. In this architecture, we use a novel hyper-fuzzy partition metric to improve the traditional common-used Euclidean norm metric clustering method. Since one fuzzy rule describes one pattern feature and implies the detection of one cluster center, it is encouraged to decrease the number of fuzzy rules with the hyper-fuzzy partition metric. According to the adaptive particle swarm optimization, it is very suitable to manage the clustering task for a complex, irregular, and high dimensional data set. To demonstrate the robustness of the proposed adaptive hyper-fuzzy partition particle swarm optimization clustering algorithms, various clustering simulations are experimentally compared with K-means and fuzzy c-means learning methods.     

Integrated use of fuzzy c-means and convex programming for capacitated multi-facility location problem

In this study a fuzzy c-means clustering algorithm based method is proposed for solving a capacitated multi-facility location problem of known demand points which are served from capacitated supply centres. It involves the integrated use of fuzzy c-means and convex programming. In fuzzy c-means, data points are allowed to belong to several clusters with different degrees of membership. This feature is used here to split demands between supply centers. The cluster number is determined by an incremental method that starts with two and designated when capacity of each cluster is sufficient for its demand. Finally, each group of cluster and each model are solved as a single facility location problem. Then each single facility location problem given by fuzzy c-means is solved by convex programming which optimizes transportation cost is used to fine-tune the facility location. Proposed method is applied to several facility location problems from OR library (Osman & Christofides, 1994) and compared with centre of gravity and particle swarm optimization based algorithms. Numerical results of an asphalt producer’s real-world data in Turkey are reported. Numerical results show that the proposed approach performs better than using original fuzzy c-means, integrated use of fuzzy c-means and center of gravity methods in terms of transportation costs.     

Fuzzy curve-tracing algorithm

This paper presents a fuzzy clustering algorithm for the extraction of a smooth curve from unordered noisy data. In this method, the input data are first clustered into different regions using the fuzzy c-means algorithm and each region is represented by its cluster center. Neighboring cluster centers are linked to produce a graph according to the average class membership values. Loops in the graph are removed to form a curve according to spatial relations of the cluster centers. The input samples are then reclustered using the fuzzy c-means (FCM) algorithm, with the constraint that the curve must be smooth. The method has been tested with both open and closed curves with good results.     

A Possibilistic Fuzzy c-Means Clustering Algorithm

In 1997, we proposed the fuzzy-possibilistic c-means (FPCM) model and algorithm that generated both membership and typicality values when clustering unlabeled data. FPCM constrains the typicality values so that the sum over all data points of typicalities to a cluster is one. The row sum constraint produces unrealistic typicality values for large data sets. In this paper, we propose a new model called possibilistic-fuzzy c-means (PFCM) model. PFCM produces memberships and possibilities simultaneously, along with the usual point prototypes or cluster centers for each cluster. PFCM is a hybridization of possibilistic c-means (PCM) and fuzzy c-means (FCM) that often avoids various problems of PCM, FCM and FPCM. PFCM solves the noise sensitivity defect of FCM, overcomes the coincident clusters problem of PCM and eliminates the row sum constraints of FPCM. We derive the first-order necessary conditions for extrema of the PFCM objective function, and use them as the basis for a standard alternating optimization approach to finding local minima of the PFCM objective functional. Several numerical examples are given that compare FCM and PCM to PFCM. Our examples show that PFCM compares favorably to both of the previous models. Since PFCM prototypes are less sensitive to outliers and can avoid coincident clusters, PFCM is a strong candidate for fuzzy rule-based system identification.     

Generalized weighted conditional fuzzy clustering

Fuzzy clustering helps to find natural vague boundaries in data. The fuzzy c-means method is one of the most popular clustering methods based on minimization of a criterion function. Among many existing modifications of this method, conditional or context-dependent c-means is the most interesting one. In this method, data vectors are clustered under conditions based on linguistic terms represented by fuzzy sets. This paper introduces a family of generalized weighted conditional fuzzy c-means clustering algorithms. This family include both the well-known fuzzy c-means method and the conditional fuzzy c-means method. Performance of the new clustering algorithm is experimentally compared with fuzzy c-means using synthetic data with outliers and the Box-Jenkins database.