融入分类词典的汉越混合网络神经机器翻译集外词处理方法

在神经机器翻译中,因词表受限导致的集外词问题很大程度上影响了翻译系统的准确性。对于训练语料较少的资源稀缺型语言的神经机器翻译,这种问题表现得更为严重。近几年,受到外部知识融入的启发,该文在RNNSearch模型基础上,提出了一种融入分类词典的汉越混合网络神经机器翻译集外词处理方法。对于给定的源语言句子,扫描分类词典以确定候选短语句对并标签标记,解码端利用词级组件和短语组件的混合解码网络,很好地生成单词集外词和短语集外词的翻译,从而改善汉越神经机器翻译的性能。在汉越、英越和蒙汉翻译实验上表明,该方法显著提高了准确率,对于资源稀缺型语言的神经机器翻译性能有一定的提升。

Unknown Words Processing Method for Chinese-Vietnamese Neural Machine Translation Based on Hybrid Network Integrating Classification Dictionaries

In neural machine translation, the problem of unknown words caused by limited vocabulary significantly affects the translation quality. Inspired by the integration of external knowledge, this paper investigates to improve the RNNSearch NMT by incorporating the classification dictionary, and proposes a new hybrid network to deal with the unknown words problem in the Chinese-Vietnamese neural machine translation. For source language sentence, the model scans classification dictionary to determine candidate phrase pairs and tags, the decoder uses hybrid network with both word and phrase level components to generate the translations. Experiments on Chinese-Vietnamese, English-Vietnamese and Mongolian-Chinese NMT show that this method significantly improves the translation performance.