Recent advances of low-resource neural machine translation

In recent years, neural network-based machine translation (MT) approaches have steadily superseded the statistical MT (SMT) methods, and represents the current state-of-the-art in MT research. Neural MT (NMT) is a data-driven end-to-end learning protocol whose training routine usually requires a large amount of parallel data in order to build a reasonable-quality MT system. This is particularly problematic for those language pairs that do not have enough parallel text for training. In order to counter the data sparsity problem of the NMT training, MT researchers have proposed various strategies, e.g. augmenting training data, exploiting training data from other languages, alternative learning strategies that use only monolingual data. This paper presents a survey on recent advances of NMT research from the perspective of low-resource scenarios.

     

Syntax-aware neural machine translation directed by syntactic dependency degree

Neural Computing and Applications - There are various ways to incorporate syntax knowledge into neural machine translation (NMT). However, quantifying the dependency syntactic intimacy (DSI)...     

Improving neural machine translation for low-resource Indian languages using rule-based feature extraction

Neural Computing and Applications - Languages help to unite the world socially, culturally and technologically. Different natives communicate in different languages; there is a tremendous...     

基于深度编码注意力的XLNet-Transformer汉-马低资源神经机器翻译优化方法

神经机器翻译(NMT)在多个领域应用中已取得显著成效,在大规模语料库上已充分论证其优越性。然而,在语料库资源不足的情形下,仍存在较大的改进空间。由于汉语-马来语(汉-马)平行语料的匮乏,直接导致了汉-马机器翻译的翻译效果不佳。为解决汉-马低资源机器翻译不理想的问题,提出了一种基于深度编码注意力和渐进式解冻的低资源神经机器翻译方法。首先,利用XLNet预训练模型重构编码器,在编码器中使用了XLNet动态聚合模块替代了传统编码层的输出方式,有效弥补了低资源汉-马语料匮乏的瓶颈;其次,在解码器中使用并行交叉注意力模块对传统编码-解码注意力进行了改进,提升了源词和目标词的潜在关系的捕获能力;最后,对提出模型采用渐进式解冻训练策略,最大化释放了模型的性能。实验结果表明,提出方法在小规模的汉-马数据集上得到了显著的性能提升,验证了方法的有效性,对比其他的低资源NMT方法,所提方法结构更为精简,并改进了编码器和解码器,翻译效果提升更加显著,为应对低资源机器翻译提供了有效的策略与启示。     

基于同义词数据增强的汉越神经机器翻译方法

汉越平行语料库的资源稀缺,很大程度上影响了汉越机器翻译效果.数据增强是提升汉越机器翻译的有效途径,基于双语词典的词汇替换数据增强是当前较为流行的方法.由于汉语-越南语属于低资源语言对,双语词典难以获得,而通过单语词向量获取低频词的同义词较为容易.因此,提出一种基于低频词的同义词替换的数据增强方法.该方法利用小规模的平行...     

N-Best句法知识增强的统计机器翻译预调序模型

源语言和目标语言的句法异构性对统计机器翻译（SMT）性能有重要影响。在基于短语的汉英统计机器翻译基础上,提出了一种基于N-best句法知识增强的源语言预调序方法。首先对源语言输入句子进行N-best句法分析,计算统计概率得到高可靠性子树结构,再根据词对齐信息从可靠性子树结构中抽取初始调序规则集。两种优化策略用于对初始规则集进行优化：基于中英文句法知识规则推导筛选和规则概率阈值控制机制。然后为减少短语内部调序,保证短语局部流利性,采用源语言短语翻译表为约束,使调序控制在短语块之间进行。最后根据获取的优化规则集和短语表约束条件对源语言端句子的句法分析树进行预调序。在基于NIST 2005和2008测试数据集上的汉英统计机器翻译实验结果表明,所提基于N-best句法知识增强的统计机器翻译预调序方法相对于基线系统,自动评价准则BLEU得分分别提高了0.68和0.83。     

句法调序的统计机器翻译方法研究

为解决基于短语统计机器翻译存在的调序能力不足的问题,尝试利用句法分析器对基于短语统计机器翻译的输入汉语句子进行句法分析,然后利用转换器进行调序操作,并对部分类型短语进行预先翻译,然后再利用基于短语统计机器翻译的解码器进行翻译。重点测试了汉语中“的”字引导的复杂定语调序、介词短语、特定搭配短语、方位词短语的调序及预翻译产生的效果。实验结果表明,这些调序及预翻译操作可以显著地提高基于短语的统计机器翻译的英文译文结果的BLEU值。     

信息传递增强的神经机器翻译

神经机器翻译领域中多层神经网络结构能够显著提升翻译效果,但是多层神经网络结构存在信息传递的退化问题。为了缓解这一问题,提出了层间和子层间信息融合传递增强的方法,增强多层神经网络的层与层之间信息传递的能力。通过引入“保留门”机制来控制融合信息的传递权重,将融合信息与当前层的输出信息连接共同作为下一层的输入,使得信息传递更加充分。在目前最先进的多层神经网络Transformer上进行相关的实验,在中英和德英翻译任务上的实验结果表明,该信息传递增强方法相比于基线系统,BLEU得分分别提高了0.66和0.42。     

Dual contextual module for neural machine translation

Machine Translation - Self-attention-based encoder-decoder frameworks have drawn increasing attention in recent years. The self-attention mechanism generates contextual representations by attending...     

基于非自回归方法的维汉神经机器翻译

现有的基于循环神经网络、卷积神经网络和Transformer的自回归翻译模型,虽然都具有良好的翻译性能,但由于解码并行性较低导致了翻译速度慢的问题,针对这个问题提出一种基于非自回归模型的优化学习率策略的方法。在基于迭代优化的非自回归序列模型的基础上,改变学习率调节方法,即把warm up替换为liner annealing方法。首先评估出liner annealing优于warm up方法,然后将liner annealing应用于非自回归序列模型以得到翻译质量和解码速度的最优平衡,最后将该方法与自回归模型的方法作对比。实验结果表明该方法相较于自回归模型Transformer,当解码速度提升1.74倍时,翻译质量的双语评估替换（BLEU）分数值为41.31,可达到Transformer的95.34%。由此可见,采用liner annealing的非自回归序列模型,在降低少许翻译质量的条件下,能够有效地提升解码速度,适用于对翻译速度需求迫切的平台。     

Recent advances in Apertium,a free/open-source rule-based machine translation platform for low-resource languages

This paper presents an overview of Apertium, a free and open-source rule-based machine translation platform. Translation in Apertium happens through a pipeline of modular tools, and the platform continues to be improved as more language pairs are added. Several advances have been implemented since the last publication, including some new optional modules: a module that allows rules to process recursive structures at the structural transfer stage, a module that deals with contiguous and discontiguous multi-word expressions, and a module that resolves anaphora to aid translation. Also highlighted is the hybridisation of Apertium through statistical modules that augment the pipeline, and statistical methods that augment existing modules. This includes morphological disambiguation, weighted structural transfer, and lexical selection modules that learn from limited data. The paper also discusses how a platform like Apertium can be a critical part of access to language technology for so-called low-resource languages, which might be ignored or deemed unapproachable by popular corpus-based translation technologies. Finally, the paper presents some of the released and unreleased language pairs, concluding with a brief look at some supplementary Apertium tools that prove valuable to users as well as language developers. All Apertium-related code, including language data, is free/open-source and available at https://github.com/apertium.

     

基于预训练的无监督神经机器翻译模型研究

依赖于大规模的平行语料库,神经机器翻译在某些语言对上已经取得了巨大的成功。无监督神经机器翻译UNMT又在一定程度上解决了高质量平行语料库难以获取的问题。最近的研究表明,跨语言模型预训练能够显著提高UNMT的翻译性能,其使用大规模的单语语料库在跨语言场景中对深层次上下文信息进行建模,获得了显著的效果。进一步探究基于跨语言预训练的UNMT,提出了几种改进模型训练的方法,针对在预训练之后UNMT模型参数初始化质量不平衡的问题,提出二次预训练语言模型和利用预训练模型的自注意力机制层优化UNMT模型的上下文注意力机制层2种方法。同时,针对UNMT中反向翻译方法缺乏指导的问题,尝试将Teacher-Student框架融入到UNMT的任务中。实验结果表明,在不同语言对上与基准系统相比,本文的方法最高取得了0.8～2.08个百分点的双语互译评估(BLEU)值的提升。     

Dual-level interactive multimodal-mixup encoder for multi-modal neural machine translation

Applied Intelligence - Multi-modal neural machine translation (MNMT), which mainly focuses on the use of image information to guide text translation. Recent MNMT approaches have been shown that...     

Supervised machine learning using encrypted training data

International Journal of Information Security - Preservation of privacy in data mining and machine learning has emerged as an absolute prerequisite in many practical scenarios, especially when the...     

基于递归神经网络的英译汉机器翻译模型设计与实现

在自然语言处理领域,递归神经网络在机器翻译中的应用越来越广泛;除了其他语言外,汉语中还包含大量的词汇,提高英译汉的机器翻译质量是对汉语处理的一个重要贡献;设计了一个英汉机器翻译系统的模型,该系统使用基于知识的上下文向量来映射英语和汉语单词,采用编解码递归神经网络实现;对基于激活函数模型的性能进行了测试,测试结果表明,编码器层的线性激活函数和解码器层的双曲正切激活函数性能最好;从GRU和LSTM层的执行情况来看,GRU的性能优于LSTM;注意层采用softmax和sigmoid激活函数进行设置,该模型的方法在交叉熵损失度量方面优于现有的系统.     

Evaluating machine translation with LFG dependencies

In this paper we show how labelled dependencies produced by a Lexical-Functional Grammar parser can be used in Machine Translation evaluation. In contrast to most popular evaluation metrics based on surface string comparison, our dependency-based method does not unfairly penalize perfectly valid syntactic variations in the translation, shows less bias towards statistical models, and the addition of WordNet provides a way to accommodate lexical differences. In comparison with other metrics on a Chinese–English newswire text, our method obtains high correlation with human scores, both on a segment and system level.     

基于模糊逻辑NSGA-Ⅲ的开关磁阻发电机多目标优化算法

低资源型的汉越神经机器翻译中,数据稀疏问题是影响翻译性能的主要原因,目前缓解该问题的途径之一是通过语料扩充方法生成伪平行数据,并用于机器翻译模型的训练,伪平行数据生成方法主要有基于词的替换、单语数据回译和枢轴翻译3种.目前的研究集中于3种方法的单独使用,缺少方法间融合利用方面的研究工作,针对此问题,提出了融入双语词典的正反向枢轴方法,利用英语作为枢轴语言,在汉到英到越正向枢轴的基础上,融入利用稀有词构建的汉-英和英-越双语词典,将汉语单语数据通过模型翻译成英语数据,再利用英-越模型将其翻译成越南语数据,其次进行越到英到汉反向枢轴翻译将越南语单语数据翻译为汉语,以此在2个方向上生成汉越伪平行数据,并利用语言模型对生成的伪平行数据进行筛选.汉-越翻译任务上的实验结果表明,提出的融入双语词典的正反向枢轴方法,能够产生更优的伪平行语料,进而显著提升汉越神经机器翻译任务的性能.