一种海量数据快速聚类算法

为满足海量数据处理要求，提出了一种基于网格的K-means快速聚类算法（SPGK）.设计基于网格质心的聚类簇个数选取算法，对数据进行网格划分得到每个网格的质心，将质心作为K-means聚类的样本点，从而减少K-means的欧氏距离计算次数.该算法基于Spark平台实现并行计算，进一步地提高了算法的运行效率.SPGK不但能够获得良好的聚类效果，而且缩减了欧氏距离计算次数，适用于海量数据的快速聚类.在千万级数据集上的实验结果表明，SPGK的性能明显优于现有的K-means++和基于K均值聚类的递归划分方法.

A Fast Clustering Algorithm for Massive Data

To meet the requirements of massive data processing, a grid-based K-means fast clustering algorithm (SPGK) is proposed. Selection for optimal clustering initial point and the number of clusters algorithm is presented. The grids of different clusters are meshed to obtain the centroid of each grid. These centroid points are used as sample points for K-means clustering, thereby reducing the number of Euclidean distance calculations of K-means. SPGK realizes parallel computation based on Spark platform, which further improves the running efficiency of the algorithm. SPGK not only obtains good clustering effect but also greatly reduces the number of Euclidean distance calculations, which is suitable for fast clustering of mass data. With 10 millions of data, the experiments show that SPGK is superior to the existing K-means++ and recursive partition based K-means clustering algorithms obviously.