Monte Carlo comparison of six hierarchical clustering methods on random data

There is mounting evidence to suggest that the complete linkage method does the best clustering job among all hierarchical agglomerative techniques, particularly with respect to misclassification in samples from known multivariate normal distributions. However, clustering methods are notorious for discovering clusters on random data sets also. We compare six agglomerative hierarchical methods on univariate random data from uniform and standard normal distributions and find that the complete linkage method generally is best in not discovering false clusters. The criterion is the ratio of number of within-cluster distances to number of all distances at most equal to the maximum within-cluster distance.     

Hierarchical clustering of time series data with parametric derivative dynamic time warping

Dynamic Time Warping (DTW) is a popular and efficient distance measure used in classification and clustering algorithms applied to time series data. By computing the DTW distance not on raw data but on the time series of the (first, discrete) derivative of the data, we obtain the so-called Derivative Dynamic Time Warping (DDTW) distance measure. DDTW, used alone, is usually inefficient, but there exist datasets on which DDTW gives good results, sometimes much better than DTW. To improve the performance of the two distance measures, we can combine them into a new single (parametric) distance function. The literature contains examples of the combining of DTW and DDTW in algorithms for supervised classification of time series data. In this paper, we demonstrate that combination of DTW and DDTW can also be applied in a method of time series clustering (unsupervised classification). In particular, we focus on a hierarchical clustering (with average linkage) of univariate (one-dimensional) time series data. We construct a new parametric distance function, combining DTW and DDTW, where a single real number parameter controls the contribution of each of the two measures to the total value of the combined distances. The parameter is tuned in the initial phase of the clustering algorithm. Using this technique in clustering methods requires a different approach (to address certain specific problems) than for supervised methods. In the clustering process we use three internal cluster validation measures (measures which do not use labels) and three external cluster validation measures (measures which do use clustering data labels). Internal measures are used to select an optimal value of the parameter of the algorithm, where external measures give information about the overall performance of the new method and enable comparison with other distance functions. Computational experiments are performed on a large real-world data base (UCR Time Series Classification Archive: 84 datasets) from a very broad range of fields, including medicine, finance, multimedia and engineering. The experimental results demonstrate the effectiveness of the proposed approach for hierarchical clustering of time series data. The method with the new parametric distance function outperforms DTW (and DDTW) on the data base used. The results are confirmed by graphical and statistical comparison.     

A methodology for clustering XML documents by structure

The processing and management of XML data are popular research issues. However, operations based on the structure of XML data have not received strong attention. These operations involve, among others, the grouping of structurally similar XML documents. Such grouping results from the application of clustering methods with distances that estimate the similarity between tree structures. This paper presents a framework for clustering XML documents by structure. Modeling the XML documents as rooted ordered labeled trees, we study the usage of structural distance metrics in hierarchical clustering algorithms to detect groups of structurally similar XML documents. We suggest the usage of structural summaries for trees to improve the performance of the distance calculation and at the same time to maintain or even improve its quality. Our approach is tested using a prototype testbed.     

Parallel self-organising hierarchical neural network based estimation of degree of voltage insecurity

Increased interconnections and loading of power systems, sometimes, lead to insecure operation. Since insecure cases often represent the most severe threats to secure system operation, it is important that the user be provided with a measure for quantifying the severity of the cases both in planning and operational stages of a power system. The Euclidean distance to the closest secure operating point has been used as a measure of the degree of insecurity. Recently, artificial neural networks are proposed increasingly for complex and time-consuming problems of power system. This paper presents a parallel self-organised hierarchical neural network based approach for estimation of the degree of voltage insecurity. Angular distance based clustering is used to select the input features. The proposed method has been tested on IEEE 30-bus system and a practical 75-bus Indian system and found to be suitable for real time implementation in Energy management centre.     

交通流时间序列模式相似性度量法

针对交通流时间序列具有高维、高噪声的特性,设计了基于趋势变动、拟合优度和最小距离和百分比原则的联机分割算法用于时间序列维约简。对分割后的时间序列进行5元组分段线性表示,并据此定义五种常见的时间序列形状相似性距离。使用分层聚类算法分析它们在不同的交通流状态辨识中的效果,以此确定交通流时间序列的模式相似性度量方法。以上海南北高架东侧间部分路段固定线圈检测数据为例进行了实证分析,最终确定模式距离与欧氏距离组合方式为交通时序模式相似性度量的最佳方法。     

一种有效的K-means聚类中心初始化方法*

传统K-means算法由于随机选取初始聚类中心,使得聚类结果波动性大;已有的最大最小距离法选取初始聚类中心过于稠密,容易造成聚类冲突现象。针对以上问题,对最大最小距离法进行了改进,提出了最大距离积法。该方法在基于密度概念的基础上,选取到所有已初始化聚类中心距离乘积最大的高密度点作为当前聚类中心。理论分析与对比实验结果表明,此方法相对于传统K-means 算法和最大最小距离法有更快的收敛速度、更高的准确率和更强的稳定性。     

OWA-based linkage method in hierarchical clustering: Application on phylogenetic trees

The linkage methods are mostly used in hierarchical clustering. In this paper, we integrate Ordered Weighted Averaging (OWA) operator with hierarchical clustering in order to find distances between clusters. In case of using OWA operator in order to find distance between clusters, OWA acts as a generalized case of single linkage, complete linkage, and average linkage methods. In order to illustrate the proposed method, we handle a phylogenetic tree constructed by hierarchical clustering of protein sequences. To illustrate the efficiency of the method, we use 2D-data set. We obtain graphs demonstrating the relationships of the clusters and we calculate the root-mean-square standard deviation (RMSSDT) and R-squared (RS) validity indices, respectively, which are frequently used to evaluate results of the hierarchical clustering algorithms.     

Long distance bigram models applied to word clustering

Two novel word clustering techniques are proposed which employ long distance bigram language models. The first technique is built on a hierarchical clustering algorithm and minimizes the sum of Mahalanobis distances of all words after a cluster merger from the centroid of the class created by merging. The second technique resorts to probabilistic latent semantic analysis (PLSA). Next, interpolated long distance bigrams are considered in the context of the aforementioned clustering techniques. Experiments conducted on the English Gigaword corpus (second edition) demonstrate that: (1) the long distance bigrams, when employed in the two clustering techniques under study, yield word clusters of better quality than the baseline bigrams; (2) the interpolated long distance bigrams outperform the long distance bigrams in the same respect; (3) the long distance bigrams perform better than the bigrams, which incorporate trigger-pairs selected at various distances; and (4) the best word clustering is achieved by the PLSA that employs interpolated long distance bigrams. Both proposed techniques outperform spectral clustering based on k-means. To assess objectively the quality of the created clusters, relative cluster validity indices are estimated as well as the average cluster sense precision, the average cluster sense recall, and the F-measure are computed by exploiting ground truth extracted from the WordNet.     

Fast approximate matching of binary codes with distinctive bits

Although the distance between binary codes can be computed fast in Hamming space, linear search is not practical for large scale datasets. Therefore attention has been paid to the efficiency of performing approximate nearest neighbor search, in which hierarchical clustering trees (HCT) are widely used. However, HCT select cluster centers randomly and build indexes with the entire binary code, this degrades search performance. In this paper, we first propose a new clustering algorithm, which chooses cluster centers on the basis of relative distances and uses a more homogeneous partition of the dataset than HCT has to build the hierarchical clustering trees. Then, we present an algorithm to compress binary codes by extracting distinctive bits according to the standard deviation of each bit. Consequently, a new index is proposed using compressed binary codes based on hierarchical decomposition of binary spaces. Experiments conducted on reference datasets and a dataset of one billion binary codes demonstrate the effectiveness and efficiency of our method.     

Combining partitional and hierarchical algorithms for robust and efficient data clustering with cohesion self-merging

Data clustering has attracted a lot of research attention in the field of computational statistics and data mining. In most related studies, the dissimilarity between two clusters is defined as the distance between their centroids or the distance between two closest (or farthest) data points However, all of these measures are vulnerable to outliers and removing the outliers precisely is yet another difficult task. In view of this, we propose a new similarity measure, referred to as cohesion, to measure the intercluster distances. By using this new measure of cohesion, we have designed a two-phase clustering algorithm, called cohesion-based self-merging (abbreviated as CSM), which runs in time linear to the size of input data set. Combining the features of partitional and hierarchical clustering methods, algorithm CSM partitions the input data set into several small subclusters in the first phase and then continuously merges the subclusters based on cohesion in a hierarchical manner in the second phase. The time and the space complexities of algorithm CSM are analyzed. As shown by our performance studies, the cohesion-based clustering is very robust and possesses excellent tolerance to outliers in various workloads. More importantly, algorithm CSM is shown to be able to cluster the data sets of arbitrary shapes very efficiently and provide better clustering results than those by prior methods.     

基于层次划分的密度优化聚类算法

针对传统的聚类算法对数据集反复聚类,且在大型数据集上计算效率欠佳的问题,提出一种基于层次划分的最佳聚类数和初始聚类中心确定算法——基于层次划分密度的聚类优化(CODHD)。该算法基于层次划分,对计算过程进行研究,不需要对数据集进行反复聚类。首先,扫描数据集获得所有聚类特征的统计值;其次,自底向上地生成不同层次的数据划分,计算每个划分数据点的密度,将最大密度点定为中心点,计算中心点距离更高密度点的最小距离,以中心点密度与最小距离乘积之和的平均值为有效性指标,增量地构建一条关于不同层次划分的聚类质量曲线;最后,根据曲线的极值点对应的划分估计最佳聚类数和初始聚类中心。实验结果表明,所提CODHD算法与预处理阶段的聚类优化(COPS)算法相比,聚类准确度提高了30%,聚类算法效率至少提高14.24%。所提算法具有较强的可行性和实用性。     

基于K-Means 的无线传感网分簇算法研究*

针对传统分层路由算法存在的分簇不均匀、簇头选举不合理以及数据传输形式单一等问题,提出基于K-Means 的无线传感网改进分簇算法LEACH-KPP。首先在成簇阶段采用K-Means 算法实现均匀分簇,随后在簇头选举阶段使用改进簇头选举函数选取簇头,最后在融合数据传输阶段根据簇头与基站,簇头与簇头之间距离动态选择单跳与多跳的混合传输方式传输数据。OMNet 仿真结果与时间复杂度推导表明,LEACH-KPP延长了网络的生存周期,在节点剩余能量与后期存活数目上都优于传统分层路由算法。     

Clustering large attributed information networks: an efficient incremental computing approach

In recent years, many information networks have become available for analysis, including social networks, road networks, sensor networks, biological networks, etc. Graph clustering has shown its effectiveness in analyzing and visualizing large networks. The goal of graph clustering is to partition vertices in a large graph into clusters based on various criteria such as vertex connectivity or neighborhood similarity. Many existing graph clustering methods mainly focus on the topological structures, but largely ignore the vertex properties which are often heterogeneous. Recently, a new graph clustering algorithm, SA-cluster, has been proposed which combines structural and attribute similarities through a unified distance measure. SA-Cluster performs matrix multiplication to calculate the random walk distances between graph vertices. As part of the clustering refinement, the graph edge weights are iteratively adjusted to balance the relative importance between structural and attribute similarities. As a consequence, matrix multiplication is repeated in each iteration of the clustering process to recalculate the random walk distances which are affected by the edge weight update. In order to improve the efficiency and scalability of SA-cluster, in this paper, we propose an efficient algorithm In-Cluster to incrementally update the random walk distances given the edge weight increments. Complexity analysis is provided to estimate how much runtime cost Inc-Cluster can save. We further design parallel matrix computation techniques on a multicore architecture. Experimental results demonstrate that Inc-Cluster achieves significant speedup over SA-Cluster on large graphs, while achieving exactly the same clustering quality in terms of intra-cluster structural cohesiveness and attribute value homogeneity.     

Clustering properties of hierarchical self-organizing maps

A multilayer hierarchical self-organizing map (HSOM) is discussed as an unsupervised clustering method. The HSOM is shown to form arbitrarily complex clusters, in analogy with multilayer feedforward networks. In addition, the HSOM provides a natural measure for the distance of a point from a cluster that weighs all the points belonging to the cluster appropriately. In experiments with both artificial and real data it is demonstrated that the multilayer SOM forms clusters that match better to the desired classes than do direct SOM's, classical k-means, or Isodata algorithms.     

基于边界距离的多向量文本聚类方法

文本聚类是自然语言处理中的一项重要研究课题,主要应用于信息检索和Web挖掘等领域。其中的关键是文本的表示和聚类算法。在层次聚类的基础上,提出了一种新的基于边界距离的层次聚类算法,该方法通过选择两个类间边缘样本点的距离作为类间距离,有效地利用类的边界信息,提高类间距离计算的准确性。综合考虑不同词性特征对文本的贡献,采用多向量模型对文本进行表示。不同文本集上的实验表明,基于边界距离的多向量文本聚类算法取得了较好的性能。     

A Multi-Resolution Content-Based Retrieval Approach for Geographic Images

Current retrieval methods in geographic image databases use only pixel-by-pixel spectral information. Texture is an important property of geographical images that can improve retrieval effectiveness and efficiency. In this paper, we present a content-based retrieval approach that utilizes the texture features of geographical images. Various texture features are extracted using wavelet transforms. Based on the texture features, we design a hierarchical approach to cluster geographical images for effective and efficient retrieval, measuring distances between feature vectors in the feature space. Using wavelet-based multi-resolution decomposition, two different sets of texture features are formulated for clustering. For each feature set, different distance measurement techniques are designed and experimented for clustering images in a database. The experimental results demonstrate that the retrieval efficiency and effectiveness improve when our clustering approach is used.     

一种抗噪的移动时间势能聚类算法

移动时间层次聚类是一种势能聚类算法,具有较好的聚类效果,但该算法无法识别数据集中存在的噪声数据点。为此,提出一种抗噪的移动时间势能聚类算法。通过各个数据点的势能值以及数据点之间的相似度找到各个数据点的父节点,计算各数据点到父节点的距离,按照该距离以及数据点的势能得到λ值,并依照λ值大小构造递增曲线,通过递增曲线中的拐点来识别出噪声点,将噪声数据归到新的类簇中,对去除噪声点后的数据集,根据数据点与父节点的距离进行层次聚类来获得聚类结果。实验结果表明,该算法能够识别出数据集中的噪声数据点,从而得到更优的聚类效果。     

基于多语义因子分层聚类的文本特征提取方法

针对同类文本中提取的关键词形式多样,且在相似性与相关性上具有模糊关系,提出一种对词语进行分层聚类的文本特征提取方法。该方法在考虑文本间相同词贡献文本相似度的前提下,结合词语相似性与相关性作为语义距离,并根据该语义距离的不同,引入分层聚类并赋予不同聚类权值的方法,最终得到以词和簇共同作为特征单元的带有聚类权值的向量空间模型。引入了word2vec训练词向量得到文本相似度,并根据Skip-Gram+Huffman Softmax模型的算法特点,运用点互信息公式准确获取词语间的相关度。通过文本的分类实验表明,所提出的方法较目前常用的仅使用相似度单层聚类后再统计的方法,能更有效地提高文本特征提取的准确性。     

Genetic clustering of social networks using random walks

In the era of globalization, traditional theories and models of social systems are shifting their focus from isolation and independence to networks and connectedness. Analyzing these new complex social models is a growing, and computationally demanding area of research. In this study, we investigate the integration of genetic algorithms (GAs) with a random-walk-based distance measure to find subgroups in social networks. We test our approach by synthetically generating realistic social network data sets. Our clustering experiments using random-walk-based distances reveal exceptionally accurate results compared with the experiments using Euclidean distances.     

基于层次Voronoi图的点群相似度算法

通过对空间点群的自适应聚类方法构建层次Voronoi图,以此层次Voronoi图为切入点,计算点群的拓扑、密度和范围的相似度,结合有关标准差的数理统计方法,计算角度、距离的相似度。在各维度的相似度基础上,使用其几何平均值作为点群整体相似度的度量标准,优化点群相似度的计算方法,并通过实验证明算法的可行性