基于词向量的藏文语义相似词知识库构建

词向量在自然语言处理研究的各个领域发挥着重要作用。该文从语言学角度出发,讨论了词向量技术与语言学理论的关系;根据词向量的特征,提出利用藏文词向量构建语义相似词知识库。该文以哈尔滨工业大学的《词林》为基础,通过汉藏双语词典对译,在获取对译词的词向量的基础上,计算对译词的词向量与原子词群平均词向量的差值,利用不同的差值,自动筛选出与原子词群语义相似度较小的词。该文分别以藏文的词和音节为单位计算词向量,自动筛出不属于原子词群的词,通过对自动筛选结果与人工筛选结果对比,发现两者具有较高的一致性,这说明词向量计算结果与人的语言直觉具有较高的一致性。总体来说,该文所采用的方法有助于提高藏文语义相似词知识库构建效率。

Construction of Knowledge Base of Semantic Similar Tibetan Words Based on Word Vectors

Word vectors play an important role in various fields of natural language processing. This paper tries to reveal the relationship between word vector technology and linguistic theory. Based on the features of word vectors, this paper proposes an approach to construct knowledge base of semantic similar Tibetan words. Based on the Chinese <Cilin> dictionary, published by Harbin University of Technology, we compute the differences between every word vector and the average word vectors of the atomic word group. With the help of Chinese-Tibetan bilingual dictionary, we deploy such differences to select the similar words from word vectors by Tibetan words and Tibetan syllables, respectively. Compared with those of manual verification, we find that the results of word vector computing are consistent with human language intuition. This approach may improve the efficiency of constructing Tibetan knowledge base of semantic similar words.