基于词汇语义信息的文本相似度计算

传统的文本相似度计算大多基于词匹配的方法，忽略了词汇语义信息，计算结果很大程度上取决于文本的词汇重复率。虽然分布式词向量可以有效表达词汇语义关系，但目前基于词向量的文本处理方法大都通过词汇串联等形式表示文本，无法体现词汇在语料库中的分布情况。 针对以上问题，本文提出了一种新的计算方法， 该方法认为基于统计的文本向量各元素之间存在相关性，且该相关性可通过词汇语义相似度表示。 因此， 本文利用词汇相似度改进了基于余弦公式的文本相似度计算方法。 实验表明该方法在 F1 值和准确度评价标准上优于其他方法。

Text similarity computing based on lexical semantic information

Traditional text similarity computation usually bases on word matching, which ignores the semantic information of the words, and the calculation results are limited by the repetition rate of the two texts. The distributed word vectors can effectively express semantic relations between words, but the text processing method based on word vector mostly express text by vocabulary series. In order to solve these problems, this paper proposes a new method to calculate the similarity of text. The method considers that there are correlations among the elements of the text vector. The correlations can be expressed by the semantic similarity of words. Therefore, the word similarity is used to improved cosine formula. We compared our method with other three methods on three popular datasets. The experimental results show that the proposed method outperforms other methods in F1 value and accuracy evaluation criteria.