基于TextRank算法和互信息相似度的维吾尔文关键词提取及文本分类

针对维吾尔语文本的分类问题,提出一种基于TextRank算法和互信息相似度的维吾尔文关键词提取及文本分类方法。首先,对输入文本进行预处理,滤除非维吾尔语的字符和停用词;然后,利用词语语义相似度、词语位置和词频重要性加权的TextRank算法提取文本关键词集合;最后,根据互信息相似度度量,计算输入文本关键词集和各类关键词集的相似度,最终实现文本的分类。实验结果表明,该方案能够 提取出具有较高识别度的关键词,当关键词集大小为1250时,平均分类率达到了91.2%。

Uyghur Keyword Extraction and Text Classification Based on TextRank Algorithm and Mutual Information Similarity

This paper proposed Uyghur keyword extraction and text classification scheme based on TextRank algorithm and mutual information similarity for the issues of classification in Uyghur language text.Firstly,the input document is pre-processed to filter out non-Uyghur characters and stop words.Then,keywords set in the text is extracted through using the TextRank algorithm which is weighted by semantic similarity of words,position of words and importance of frequency.Finally,the similarity between keyword sets in the input text and a variety of keyword sets is measured according to the mutual information similarity,and the text classification is realized.The experimental results show that this scheme can efficiently extract the keywords,and the average classification rate reaches 91.2% when the set size is 1250.