基于相对熵的KNN文本分类方法的研究

摘要:为提高处理文本相似度的效果,提出了一种基于相对熵度量文本差异的KNN算法.该算法首先对文本进行预处理(分字与删去停用字)和构建特征字字典; 然后计算训练集中所有文本特征字的概率,并组成训练集(特征字概率矩阵); 最后计算预测文本的特征字概率向量,并通过计算和统计K个预测文本与训练集文本间相对熵最小的文本类别个数后将数目最多的类别作为测试样本的类别.实验结果表明,该算法的分类效果不仅显著优于传统KNN、SVM、Decision Tree、朴素Bayes算法的分类效果,且在小样本数据情况下

Research on text classification of K-nearest neighbor algorithm based on relative entropy

To improve the effectiveness of processing text similarity, a KNN algorithm based on the relative entropy measure of text feature differences was proposed in this paper. Firstly, the algorithm preprocessed the text, including character separation and deletion of stop characters, and constructed a feature character dictionary. Then the probabilities of all the text feature characters in the training set were calculated, and the training set(probability matrix of feature character)was formed. Finally, we calculated the probability vector of feature characters of the predicted text, and counted the number of text categories with the lowest relative entropy between the K predicted texts and the training set, and used the category with the highest number as the category of the test sample. The experimental results show that the classification effect of this algorithm is not only significantly better than that of the traditional KNN, SVM, Decision Tree, and Naive Bayes, but also significantly better than that of RNN algorithm in the case of small sample data.