改进的维吾尔语Web文本后缀树聚类

该文提出了改进的维吾尔语Web文本后缀树聚类算法STCU,其中后缀树的构建以维吾尔语句子为基本单位。针对维吾尔语语言和Web文本特点,文中对词语进行词干提取,构建了维吾尔语绝对停用词表和相对停用词表,采用文档频率和词性结合的方法提取关键短语,改进了合并基类的二进制方法,根据语料类别数自动调整聚类类别阈值,利用最一般短语对聚类类别进行描述,有效地改善了文本聚类的质量。与传统的后缀树聚类算法相比,聚类全面率提高了44.51%,聚类准确率提高了11.74%,错误率降低了0.94%。实验结果表明 改进的后缀树算法在Web文本聚类的精度和效率方面具有较强的优越性。

An Improved Uyghur Web Text Clustering Based on Suffix Tree

The paper proposes an improved suffix tree clustering algorithm for Uyghur Web text (STCU), with the Uyghur word as the basic unit in the construction the suffix tree. According to the characteristics of Uyghur and Web texts, we design the Uyghur word stemmer, and construct Uyghur absolute stop word table and relative stop word table. We adopt the document frequency and part-of-speech information to extract key phrases, and then automatically adjust clustering threshold according to the number of Web corpus. Finally, we utilize the most general phrases to describe clustering category information, effectively improving the quality of clustering results. Compared to the traditional suffix tree clustering, the error rate has dropped 0.94%, and in turn, the overall rate and the precision have improved by 44.51% and 11.74%, respectively.
Key wordsUyghur; suffix tree; phrase clustering; stop word list; document frequency