一种基于语义内积空间模型的文本聚类算法

现有数据聚类方法在处理文本数据,尤其是短文本数据时,由于没有考虑词之间潜在存在的相似情况,因此导致聚类效果不理想.文中针对文本数据高维度和稀疏空间的特点,提出了一种基于语义内积空间模型的文本聚类算法.算法首先利用内积空间的定义建立了针对中文概念、词和文本的相似度度量方法,然后从理论上进行了分析.最后通过一个两阶段处理过程,即向下分裂和向上聚合,完成文本数据的聚类.该方法成功用于中文短文本数据的聚类.实验表明相对于传统方法,文中提供的方法聚类质量更好.

A Novel Text Clustering Algorithm Based on Inner Product Space Model of Semantic

Due to lack considering the latent similarity information among words, the clustering result using exist clustering algorithms in processing text data, especially in processing short text data, is not ideal. Considering the text characteristic of high dimensions and sparse space, this paper proposes a novel text clustering algorithm based on semantic inner space model. The paper creates similarity method among Chinese concepts, words and text based on the definition of inner space at first, and then analyzes systematically the algorithm in theory. Through a two phrase processes, i.e. top-down "divide" phase and a bottom-up "merge" phase, it finishes the clustering of text data. The method has been applied into the data clustering of Chinese short documents. Extensive experiments show that the method is better than traditional algorithms.