层次非负矩阵分解及在文本聚类中的应用

文本聚类的目标是把数据集中内容相似的文档归为一类,而使内容不同的文档分开。目前针对不同领域的需求,多种解决聚类问题的算法应运而生。然而,由于文本数据本身固有的复杂特点,如海量、高维、稀疏等,使得对海量文本数据的聚类仍然是一个棘手的问题。提出了层次非负矩阵分解聚类方法,该方法不但保留了非负矩阵分解的优点,如同步识别文档类别和找出类别本质特征,而且能够展现类别间的层次结构。这种类别层次结构在网页预览等应用中是非常有用的。在真实数据集20Newsgroups和Reuters-RCV1上的实验结果表明,层次非负矩阵分解相比已有的方法更有效。     

A genetic algorithm approach to cluster analysis

A common problem in the social and agricultural sciences is to find clusters in experimental data; the standard attack is a deterministic search terminating in a locally optimal clustering. We propose here a genetic algorithm (GA) for performing cluster analysis. GAs have been used profitably in a variety of contexts in which it is either impractical or impossible to directly solve for a globally optimal solution to complex numerical problems. In the present case, our GA clustering technique attempted to maximize a variance-ratio (VR) based goodness-of-fit criterion defined in terms of external cluster isolation and internal cluster homogeneity. Although our GA-based clustering algorithm cannot guarantee to recover the cluster solution that exhibits the global maximum of this fitness function, it does explicitly work toward this goal (in marked contrast to existing clustering algorithms, especially hierarchical agglomerative ones such as Ward's method). Using both constrained and unconstrained simulated datasets, Monte Carlo results showed that in some conditions the genetic clustering algorithm did indeed surpass the performance of conventional clustering techniques (Ward's and K-means) in terms of an internal (VR) criterion. Suggestions for future refinement and study are offered.     

A new clustering method based on the bio-inspired cuttlefish optimization algorithm

Most of the well-known clustering methods based on distance measures, distance metrics and similarity functions have the main problem of getting stuck in the local optima and their performance strongly depends on the initial values of the cluster centers. This paper presents a new approach to enhance the clustering problems with the bio-inspired Cuttlefish Algorithm (CFA) by searching the best cluster centers that can minimize the clustering metrics. Various UCI Machine Learning Repository datasets are used to test and evaluate the performance of the proposed method. For the sake of comparison, we have also analysed several algorithms such as K-means, Genetic Algorithm and the Particle Swarm Optimization (PSO) Algorithm. The simulations and obtained results demonstrate that the performance of the proposed CFA-Clustering method is superior to the other counterpart algorithms in most cases. Therefore, the CFA can be considered as an alternative stochastic method to solve clustering problems.     

A novel graph-based clustering method using noise cutting

Recently, many methods have appeared in the field of cluster analysis. Most existing clustering algorithms have considerable limitations in dealing with local and nonlinear data patterns. Algorithms based on graphs provide good results for this problem. However, some widely used graph-based clustering methods, such as spectral clustering algorithms, are sensitive to noise and outliers. In this paper, a cut-point clustering algorithm (CutPC) based on a natural neighbor graph is proposed. The CutPC method performs noise cutting when a cut-point value is above the critical value. Normally, the method can automatically identify clusters with arbitrary shapes and detect outliers without any prior knowledge or preparatory parameter settings. The user can also adjust a coefficient to adapt clustering solutions for particular problems better. Experimental results on various synthetic and real-world datasets demonstrate the obvious superiority of CutPC compared with k-means, DBSCAN, DPC, SC, and DCore.     

Shared farthest neighbor approach to clustering of high dimensionality, low cardinality data

Clustering algorithms are routinely used in biomedical disciplines, and are a basic tool in bioinformatics. Depending on the task at hand, there are two most popular options, the central partitional techniques and the agglomerative hierarchical clustering techniques and their derivatives. These methods are well studied and well established. However, both categories have some drawbacks related to data dimensionality (for partitional algorithms) and to the bottom-up structure (for hierarchical agglomerative algorithms). To overcome these limitations, motivated by the problem of gene expression analysis with DNA microarrays, we present a hierarchical clustering algorithm based on a completely different principle, which is the analysis of shared farthest neighbors. We present a framework for clustering using ranks and indexes, and introduce the shared farthest neighbors (SFN) clustering criterion. We illustrate the properties of the method and present experimental results on different data sets, using the strategy of evaluating data clustering by extrinsic knowledge given by class labels.     

A niching genetic k-means algorithm and its applications to gene expression data

Partitional clustering is a common approach to cluster analysis. Although many algorithms have been proposed, partitional clustering remains a challenging problem with respect to the reliability and efficiency of recovering high quality solutions in terms of its criterion functions. In this paper, we propose a niching genetic k-means algorithm (NGKA) for partitional clustering, which aims at reliably and efficiently identifying high quality solutions in terms of the sum of squared errors criterion. Within the NGKA, we design a niching method, which encourages mating among similar clustering solutions while allowing for some competitions among dissimilar solutions, and integrate it into a genetic algorithm to prevent premature convergence during the evolutionary clustering search. Further, we incorporate one step of k-means operation into the regeneration steps of the resulted niching genetic algorithm to improve its computational efficiency. The proposed algorithm was applied to cluster both simulated data and gene expression data and compared with previous work. Experimental results clear show that the NGKA is an effective clustering algorithm and outperforms two other genetic algorithm based clustering methods implemented for comparison.     

基于划分和层次的混合动态聚类算法*

针对划分聚类对初始值较为敏感以及层次聚类时间复杂度高等缺陷,提出了一种基于划分和层次的混合动态聚类算法HDC-PH。该算法首先使用划分聚类快速生成一定数量的子簇,然后以整体相似度的聚类质量评价标准来动态改变聚类数目,同时给出了聚类过程中孤立点的剔除方法。实验结果表明,HDC-PH算法的性能明显优于划分和层次算法,提高了聚类质量,并获得了更自然的聚类结果。     

MultiClusterTree: Interactive Visual Exploration of Hierarchical Clusters in Multidimensional Multivariate Data

Visual analytics of multidimensional multivariate data is a challenging task because of the difficulty in understanding metrics in attribute spaces with more than three dimensions. Frequently, the analysis goal is not to look into individual records but to understand the distribution of the records at large and to find clusters of records with similar attribute values. A large number of (typically hierarchical) clustering algorithms have been developed to group individual records to clusters of statistical significance. However, only few visualization techniques exist for further exploring and understanding the clustering results. We propose visualization and interaction methods for analyzing individual clusters as well as cluster distribution within and across levels in the cluster hierarchy. We also provide a clustering method that operates on density rather than individual records. To not restrict our search for clusters, we compute density in the given multidimensional multivariate space. Clusters are formed by areas of high density. We present an approach that automatically computes a hierarchical tree of high density clusters. To visually represent the cluster hierarchy, we present a 2D radial layout that supports an intuitive understanding of the distribution structure of the multidimensional multivariate data set. Individual clusters can be explored interactively using parallel coordinates when being selected in the cluster tree. Furthermore, we integrate circular parallel coordinates into the radial hierarchical cluster tree layout, which allows for the analysis of the overall cluster distribution. This visual representation supports the comprehension of the relations between clusters and the original attributes. The combination of the 2D radial layout and the circular parallel coordinates is used to overcome the overplotting problem of parallel coordinates when looking into data sets with many records. We apply an automatic coloring scheme based on the 2D radial layout of the hierarchical cluster tree encoding hue, saturation, and value of the HSV color space. The colors support linking the 2D radial layout to other views such as the standard parallel coordinates or, in case data is obtained from multidimensional spatial data, the distribution in object space.     

A review on particle swarm optimization algorithms and their applications to data clustering

Data clustering is one of the most popular techniques in data mining. It is a method of grouping data into clusters, in which each cluster must have data of great similarity and high dissimilarity with other cluster data. The most popular clustering algorithm K-mean and other classical algorithms suffer from disadvantages of initial centroid selection, local optima, low convergence rate problem etc. Particle Swarm Optimization (PSO) is a population based globalized search algorithm that mimics the capability (cognitive and social behavior) of swarms. PSO produces better results in complicated and multi-peak problems. This paper presents a literature survey on the PSO application in data clustering. PSO variants are also described in this paper. An attempt is made to provide a guide for the researchers who are working in the area of PSO and data clustering.     

Is VAT really single linkage in disguise?

This paper addresses the relationship between the Visual Assessment of cluster Tendency (VAT) algorithm and single linkage hierarchical clustering. We present an analytical comparison of the two algorithms in conjunction with numerical examples to show that VAT reordering of dissimilarity data is directly related to the clusters produced by single linkage hierarchical clustering. This analysis is important to understanding the underlying theory of VAT and, more generally, other algorithms that are based on VAT-ordered dissimilarity data.     

Cluster merging based on a decision threshold

Data clustering is an important unsupervised learning technique and has wide application in various fields including pattern recognition, data mining, image analysis and bioinformatics. A vast amount of clustering algorithms have been proposed in the past decades. However, existing algorithms still face many problems in practical applications. One typical problem is the parameter dependence, which means that user-specified parameters are required as input and the clustering results are influenced by these parameters. Another problem is that many algorithms are not able to generate clusters of non-spherical shapes. In this paper, a cluster merging method is proposed to solve the above-mentioned problems based on a decision threshold and the dominant sets algorithm. Firstly, the influence of similarity parameter on dominant sets clustering results is studied, and it is found that the obtained clusters become larger with the increase in similarity parameter. We analyze the reason behind this behavior and propose to generate small initial clusters in the first step and then merge the initial clusters to improve the clustering results. Specifically, we select a similarity parameter which generates small but not too small clusters. Then, we calculate pairwise merging decisions among the initial clusters and obtain a merging decision threshold. Based on this threshold, we merge the small clusters and obtain the final clustering results. Experiments on several datasets are used to validate the effectiveness of the proposed algorithm.

     

On Hierarchical Diameter-Clustering and the Supplier Problem

Given a data set in a metric space, we study the problem of hierarchical clustering to minimize the maximum cluster diameter, and the hierarchical k-supplier problem with customers arriving online. We prove that two previously known algorithms for hierarchical clustering, one (offline) due to Dasgupta and Long and the other (online) due to Charikar, Chekuri, Feder and Motwani, output essentially the same result when points are considered in the same order. We show that the analyses of both algorithms are tight and exhibit a new lower bound for hierarchical clustering. Finally we present the first constant factor approximation algorithm for the online hierarchical k-supplier problem.     

Density-based hierarchical clustering for streaming data

For streaming data that arrive continuously such as multimedia data and financial transactions, clustering algorithms are typically allowed to scan the data set only once. Existing research in this domain mainly focuses on improving the accuracy of clustering. In this paper, a novel density-based hierarchical clustering scheme for streaming data is proposed in order to improve both accuracy and effectiveness; it is based on the agglomerative clustering framework. Traditionally, clustering algorithms for streaming data often use the cluster center to represent the whole cluster when conducting cluster merging, which may lead to unsatisfactory results. We argue that even if the data set is accessed only once, some parameters, such as the variance within cluster, the intra-cluster density and the inter-cluster distance, can be calculated accurately. This may bring measurable benefits to the process of cluster merging. Furthermore, we employ a general framework that can incorporate different criteria and, given the same criteria, will produce similar clustering results for both streaming and non-streaming data. In experimental studies, the proposed method demonstrates promising results with reduced time and space complexity.     

K-Harmonic means type clustering algorithm for mixed datasets

K-means type clustering algorithms for mixed data that consists of numeric and categorical attributes suffer from cluster center initialization problem. The final clustering results depend upon the initial cluster centers. Random cluster center initialization is a popular initialization technique. However, clustering results are not consistent with different cluster center initializations. K-Harmonic means clustering algorithm tries to overcome this problem for pure numeric data. In this paper, we extend the K-Harmonic means clustering algorithm for mixed datasets. We propose a definition for a cluster center and a distance measure. These cluster centers and the distance measure are used with the cost function of K-Harmonic means clustering algorithm in the proposed algorithm. Experiments were carried out with pure categorical datasets and mixed datasets. Results suggest that the proposed clustering algorithm is quite insensitive to the cluster center initialization problem. Comparative studies with other clustering algorithms show that the proposed algorithm produce better clustering results.     

Clustering data with measurement errors

Traditional clustering methods assume that there is no measurement error, or uncertainty, associated with data. Often, however, real world applications require treatment of data that have such errors. In the presence of measurement errors, well-known clustering methods like k-means and hierarchical clustering may not produce satisfactory results.In this article, we develop a statistical model and algorithms for clustering data in the presence of errors. We assume that the errors associated with data follow a multivariate Gaussian distribution and are independent between data points. The model uses the maximum likelihood principle and provides us with a new metric for clustering. This metric is used to develop two algorithms for error-based clustering, hError and kError, that are generalizations of Ward's hierarchical and k-means clustering algorithms, respectively.We discuss types of clustering problems where error information associated with the data to be clustered is readily available and where error-based clustering is likely to be superior to clustering methods that ignore error. We focus on clustering derived data (typically parameter estimates) obtained by fitting statistical models to the observed data. We show that, for Gaussian distributed observed data, the optimal error-based clusters of derived data are the same as the maximum likelihood clusters of the observed data. We also report briefly on two applications with real-world data and a series of simulation studies using four statistical models: (1) sample averaging, (2) multiple linear regression, (3) ARIMA models for time-series, and (4) Markov chains, where error-based clustering performed significantly better than traditional clustering methods.     

Clustering data with measurement errors

Traditional clustering methods assume that there is no measurement error, or uncertainty, associated with data. Often, however, real world applications require treatment of data that have such errors. In the presence of measurement errors, well-known clustering methods like k-means and hierarchical clustering may not produce satisfactory results.In this article, we develop a statistical model and algorithms for clustering data in the presence of errors. We assume that the errors associated with data follow a multivariate Gaussian distribution and are independent between data points. The model uses the maximum likelihood principle and provides us with a new metric for clustering. This metric is used to develop two algorithms for error-based clustering, hError and kError, that are generalizations of Ward's hierarchical and k-means clustering algorithms, respectively.We discuss types of clustering problems where error information associated with the data to be clustered is readily available and where error-based clustering is likely to be superior to clustering methods that ignore error. We focus on clustering derived data (typically parameter estimates) obtained by fitting statistical models to the observed data. We show that, for Gaussian distributed observed data, the optimal error-based clusters of derived data are the same as the maximum likelihood clusters of the observed data. We also report briefly on two applications with real-world data and a series of simulation studies using four statistical models: (1) sample averaging, (2) multiple linear regression, (3) ARIMA models for time-series, and (4) Markov chains, where error-based clustering performed significantly better than traditional clustering methods.     

凝聚中心犹豫度恒定的模糊层次聚类算法

传统的模糊方法已无法解决数据本身不确定性的问题,犹豫模糊集方法却行之有效.原有的犹豫模糊层次聚类算法没有考虑犹豫模糊集对权值的影响,缺乏合理的权重计算方法,并且算法的时间复杂度和空间复杂度都为指数级.为了更有效地解决聚类分析问题,本文提出一种凝聚中心犹豫度恒定的模糊层次聚类算法(FHCA),首先设计了一种基于数据集本身...     

基于模糊商空间的聚类分析方法

在商空间理论基础上,提出了基于Fuzzy相似关系和归一化距离的聚类分析方法,用以解决复杂系统的数据结构分析问题.得到了如下结论:(1)通过引入基于Fuzzy相似关系和归一化距离的分层递阶结构,建立了严格的聚类分析理论描述;(2)给出了有效的分层递阶结构聚类的快速算法;(3)给出了两个Fuzzy相似关系或由两个归一化距离诱导的Fuzzy相似关系是同构的充分条件.其中所研究的理论和方法适应于建立在相似关系之上的任何复杂系统的数据结构分析.     

Statistical,Evolutionary, and Neurocomputing Clustering Techniques: Cluster-Based vs Object-Based Approaches

Modern day computers cannot provide optimal solution to the clustering problem. There are many clustering algorithms that attempt to provide an approximation of the optimal solution. These clustering techniques can be broadly classified into two categories. The techniques from first category directly assign objects to clusters and then analyze the resulting clusters. The methods from second category adjust representations of clusters and then determine the object assignments. In terms of disciplines, these techniques can be classified as statistical, genetic algorithms based, and neural network based. This paper reports the results of experiments comparing five different approaches: hierarchical grouping, object-based genetic algorithms, cluster-based genetic algorithms, Kohonen neural networks, and K-means method. The comparisons consist of the time requirements and within-group errors. The theoretical analyses were tested for clustering of highway sections and supermarket customers. All the techniques were applied to clustering of highway sections. The hierarchical grouping and genetic algorithms approaches were computationally infeasible for clustering a larger set of supermarket customers. Hence only Kohonen neural networks and K-means techniques were applied to the second set to confirm some of the results from previous experiments.     

Self-Organizing-Map Based Clustering Using a Local Clustering Validity Index

Classical clustering methods, such as partitioning and hierarchical clustering algorithms, often fail to deliver satisfactory results, given clusters of arbitrary shapes. Motivated by a clustering validity index based on inter-cluster and intra-cluster density, we propose that the clustering validity index be used not only globally to find optimal partitions of input data, but also locally to determine which two neighboring clusters are to be merged in a hierarchical clustering of Self-Organizing Map (SOM). A new two-level SOM-based clustering algorithm using the clustering validity index is also proposed. Experimental results on synthetic and real data sets demonstrate that the proposed clustering algorithm is able to cluster data in a better way than classical clustering algorithms on an SOM.