跨语言文献相似度的分析方法

对不同语言的句对齐文献资料进行分析,提出了基于多语主题模型的跨语言文献相似度的计算方法.首先,对收集整理的不同语言(中文、英文、韩文)文献构建数据模型,通过分词、分词结果修正及选择、词权重计算等预处理工作构造词项-文档矩阵.其次,建立多语主题语义空间,将译成3种不同语言的文献映射到语义空间,在语义空间中每一主题都由3种语言构成.最后,通过其语义空间中对应的主题计算比较不同语言间的文献相似度.实验结果显示,不同语言之间的文献相似度可以直接在语义空间中计算,且相似度计算的准确性在90%以上,验证了本文方法在跨语言文献相似度计算时的有效性.

An analysis method of cross-lingual literature similarity

We analyse different language literatures with sentence alignment and propose a cross-lingual literatures’ similarity method based on multilingual topic correlation model. In this paper, the data model for the collected different language literatures is firstly gained by term-document matrix, which is obtained by the process of words segmentation, the adjustment and selection of words segmentation results, and the weight calculation of feature words.And then, multilingual topic correlation semantic space is built. The three different language literatures are represented in the semantic space where each topic is made up of the three languages.Similarity calculation of different language literatures is completed by their correlation topic in the semantic space. Experiment results show that the similarity of different language literaturescan be calculated directly in the semantic space, the accuracy can be reached 90%, which verify the effectiveness of our method in calculating the similarity of cross-lingual literatures.