基于并行图计算的社区划分方法

以图计算形式研究社交网络由来已久，但对于如何提升图计算应用于大规模社交网络的计算速度和扩展性，一直是研究的难点。谱图论的应用为社交网络在图计算方面的研究带来新的研究热点，谱图分割为社交网络社区划分带来基于结构的支撑。为了解决谱图论在处理大规模社交网络时存在计算缓慢、内存溢出等问题，本文提出了谱聚类改进算法结合矩阵方式在并行环境下的处理方法。首先，利用Spark对网络数据进行并行化预处理，将社交网络以图结构表示，再将图转化为Spark分布式稀疏矩阵。然后，将谱聚类改进算法在Spark环境下，实现并行化社交网络社区快速划分，并以分布式方式持久化存储源数据、中间计算数据和计算结果，提高图计算在社交网络中的可靠性。最后，通过实验证明并行化图计算方法能有效提高计算速度和扩展性，支持大规模社交网络的挖掘分析，实现并行算法下高并发、高吞吐的特点。

Community partitioning method based on parallel graph calculation

It is difficult to study the speed and expansibility of how to improve the calculation and application of the graph calculation to the large-scale social network. The application of spectral theory brings new research hotspots to the research of social network in graph calculation. The spectrum segmentation is based on the structure support of social network community division. In order to solve the problem that the spectrum theory has slow computation and memory overflow when dealing with large-scale social network, this paper proposes a method of combining spectral clustering improved algorithm combined with matrix method in parallel environment. First of all, using Spark to parallelize the network data preprocessing, the social network to map structure, and then converted into Spark distributed sparse matrix. Then, the spectral clustering algorithm was used to realize the parallelization of the social network community in the Spark environment, and the storage source data, the intermediate calculation data and the calculation result were obtained in a distributed way, and the reliability of the graph calculation in the social network was improved.The Finally, it is proved that the parallel graph calculation method can effectively improve the calculation speed and expansibility, support the mining analysis of large-scale social network, and realize the characteristics of high concurrent and high throughput under the parallel algorithm.