基于Word2vec的句子语义相似度计算研究

word2vec利用深度学习的思想,可以从大规模的文本数据中自动学习数据的本质信息。因此,借助哈尔滨工业大学的LTP平台,设计利用word2vec模型将对句子的处理简化为向量空间中的向量运算,采用向量空间上的相似度表示句子语义上的相似度。此外,将句子的结构信息添加到句子相似度计算中,并就特殊句式对算法进行了改进,同时考虑到了词汇之间的句法关系。实验结果表明,该方法更准确地揭示了句子之间的语义关系,句法结构的提取和算法的改进解决了复杂句式的相似度计算问题,提高了相似度计算的准确率。

Research on Sentence Semantic Similarity Calculation Based on Word2vec

Using the idea of deep learning,word2vec can automatically learn the essential information of data from large-scale text data.Therefore,with the help of LTP platform of Harbin Institute of Technology,based on the word2vec model,the processing of the sentence is simplified as a vector in the vector space algorithm,and the similarity of vector space represents the sentence semantic similarity.In addition,the sentence structure information is added to the sentence similarity calculation,the algorithm are improved on the special sentence pattern,and the syntax relationship between words is taken into account.The experimental results show that this method is more accurately to reveal the semantic relations between sentences,syntactic structure and improved extraction algorithm also solve the problem of computing the similarity of complex sentences,finally improve the accuracy of the similarity calculation.