基于注意力卷积的神经机器翻译

现有神经机器翻译模型普遍采用的注意力机制是基于单词级别的,文中通过在注意力机制上执行多层卷积,从而将注意力机制从基于单词的级别提高到基于短语的级别。经过卷积操作后的注意力信息将愈加明显地体现出短语结构性,并被用于生成新的上下文向量,从而将新生成的上下文向量融入到神经机器翻译框架中。在大规模的中-英测试数据集上的实验结果表明,基于注意力卷积的神经机翻译模型能够很好地捕获语句中的短语结构信息,增强翻译词前后的上下文依赖关系,优化上下文向量,提高机器翻译的性能。

Neural Machine Translation Based on Attention Convolution

The attention mechanism commonly used by the existing neural machine translation is based on the word level.By creating multi-layer convolutional structure on the basis of attention mechanism,this paper improved attention mecha-nism from word-based level to phrase-based level.After convolutional operation,the attention information can reflect phrase structure more clearly and generate new context vectors.Then,the new context vectors are used to integrate into the neural machine translation framework.Experimental results on large-scale Chinese-to-English tasks show that neural machine translation based on attention convolution can effectively capture the phrasal information in statements,enhance the context dependencies of translated words,optimize the context vectors and improve the translation quality.