多策略汉日机器翻译系统中的核心技术研究

多策略的机器翻译是当今机器翻译系统的一个发展方向。该文论述了一个多策略的汉日机器翻译系统中各翻译核心子系统所使用的核心技术和算法,其中包含了使用词法分析、句法分析和语义角色标注的汉语分析子系统、利用双重索引技术的基于翻译记忆技术的机器翻译子系统、以句法树片段为模板的基于实例模式的机器翻译子系统以及综合了配价模式和断段分析的机器翻译子系统。翻译记忆子系统的测试结果表明其具有高效的特性;实例模式子系统在1 559个句子的封闭测试中达到99%的准确率,在1 500个句子的开放测试中达到85%的准确率;配价模式子系统在3 059个句子的测试中达到了89%的准确率。     

基于实例的机器翻译中实例匹配算法的研究

基于实例的机器翻译 (Example BasedMachineTranslation ,简称EBMT)通过模仿实例的翻译实现源文的翻译。在EBMT中 ,实例的匹配是关键 ,它直接关系到EBMT本身的翻译质量。文章通过对现有几类实例匹配算法的比较和研究 ,提出一种基于模式的实例匹配算法。     

Meaning preservation in Example-based Machine Translation with structural semantics

The main tasks in Example-based Machine Translation (EBMT) comprise of source text decomposition, following with translation examples matching and selection, and finally adaptation and recombination of the target translation. As the natural language is ambiguous in nature, the preservation of source text’s meaning throughout these processes is complex and challenging. A structural semantics is introduced, as an attempt towards meaning-based approach to improve the EBMT system. The structural semantics is used to support deeper semantic similarity measurement and impose structural constraints in translation examples selection. A semantic compositional structure is derived from the structural semantics of the selected translation examples. This semantic compositional structure serves as a representation structure to preserve the consistency and integrity of the input sentence’s meaning structure throughout the recombination process. In this paper, an English to Malay EBMT system is presented to demonstrate the practical application of this structural semantics. Evaluation of the translation test results shows that the new translation framework based on the structural semantics has outperformed the previous EBMT framework.     

Evaluating machine translation with LFG dependencies

In this paper we show how labelled dependencies produced by a Lexical-Functional Grammar parser can be used in Machine Translation evaluation. In contrast to most popular evaluation metrics based on surface string comparison, our dependency-based method does not unfairly penalize perfectly valid syntactic variations in the translation, shows less bias towards statistical models, and the addition of WordNet provides a way to accommodate lexical differences. In comparison with other metrics on a Chinese–English newswire text, our method obtains high correlation with human scores, both on a segment and system level.     

机器翻译测评的实践与分析

机器翻译是人工智能领域中一项有挑战性的研究课题,而对机器翻译系统的测评也越来越受到重视。为了考察现今机器翻译系统的翻译质量及其出现的问题,提出了采用大规模语料库作为测试集的方法,对六个机器翻译系统进行了半自动测评,分析了机器翻译系统在词法和句法上存在的问题,并将测评结果与NIST自动测评进行比较,分析了测评结果的客观性。     

基于序列相交的短语译文获取

短语译文获取技术是基于实例的机器翻译(EBMT)中的核心技术之一,其准确率直接影响到EBMT系统的性能。该文提出了一种基于序列相交的短语译文获取方法,该方法将句子视为词的序列,利用对中日句对齐语料库中包含待译短语的所有源语句子对应的目标语句子进行序列相交的方式,在不需要词对齐、句法分析及词典等资源的情况下,通过充分挖掘句对齐双语语料库的信息,获得高质量的短语译文。实验表明,该方法获得的短语译文准确率超过80%。     

Hybrid data-driven models of machine translation

This paper presents an extended, harmonised account of our previous work on combining subsentential alignments from phrase-based statistical machine translation (SMT) and example-based MT (EBMT) systems to create novel hybrid data-driven systems capable of outperforming the baseline SMT and EBMT systems from which they were derived. In previous work, we demonstrated that while an EBMT system is capable of outperforming a phrase-based SMT (PBSMT) system constructed from freely available resources, a hybrid ‘example-based’ SMT system incorporating marker chunks and SMT subsentential alignments is capable of outperforming both baseline translation models for French–English translation. In this paper, we show that similar gains are to be had from constructing a hybrid ‘statistical’ EBMT system. Unlike the previous research, here we use the Europarl training and test sets, which are fast becoming the standard data in the field. On these data sets, while all hybrid ‘statistical’ EBMT variants still fall short of the quality achieved by the baseline PBSMT system, we show that adding the marker chunks to create a hybrid ‘example-based’ SMT system outperforms the two baseline systems from which it is derived. Furthermore, we provide further evidence in favour of hybrid systems by adding an SMT target-language model to the EBMT system, and demonstrate that this too has a positive effect on translation quality. We also show that many of the subsentential alignments derived from the Europarl corpus are created by either the PBSMT or the EBMT system, but not by both. In sum, therefore, despite the obvious convergence of the two paradigms, the crucial differences between SMT and EBMT contribute positively to the overall translation quality. The central thesis of this paper is that any researcher who continues to develop an MT system using either of these approaches will benefit further from integrating the advantages of the other model; dogged adherence to one approach will lead to inferior systems being developed.     

一种维吾尔语句子相似度算法的研究

基于实例的机器翻译是一种重要的机器翻译技术,句子相似度的衡量是基于实例机器翻译研究中最重要的一个内容。对于基于实例的维吾尔语机器翻译研究,维吾尔语句子相似度衡量的准确性,直接影响到最后翻译结果的输出。提出了一种维吾尔语句子相似度的计算方法,采用的基于词形特征的粗选算法、散列单词倒排索引能够有效提高算法的查找速度,快速从语料库中筛选出候选句子集合;多策略精选算法中采用基于维吾尔语词频的单词区分度算法、连续单词序列抽取算法,可以有效衡量两个维吾尔语句子的相似程度,实验结果证明算法是有效的。     

基于自动句对齐的相似古文句子检索

随着语料库语言学的兴起,基于实例的机器翻译(EBMT)得到越来越多的研究。如何快速准确地构建大规模古今汉语平行语料库,以及从大量的对齐实例(句子级)中检索和输入句子最相似的源句子是基于实例的古今汉语机器翻译必须解决的问题。本文综合考虑句子长度、汉字字形、标点符号三个因素提出了古今汉语句子互译模型,基于遗传算法、动态规划算法实现了古今汉语的自动句对齐。接着为古文句子建立全文索引,基于汉字的信息熵,本文设计与实现一种高效的最相似古文句子检索算法。最后给出了自动句对齐和最相似古文句子检索的实验结果。     

基于句对质量和覆盖度的统计机器翻译训练语料选取

该文研究的目的是在待翻译文本未知的情况下,从已有的大规模平行语料中选取一个高质量的子集作为统计机器翻译系统的训练语料,以降低训练和解码代价。该文综合覆盖度和句对翻译质量两方面因素,提出一种从已有平行语料中获取高质量小规模训练子集的方法。在CWMT2008汉英翻译任务上的实验结果表明,利用本文的方法能够从现有大规模语料中选取高质量的子集,在减少80%训练语料的情况下达到与Baseline系统(使用全部训练语料)相当的翻译性能(BLEU值)。     

Machine Translation of Closed Captions

Traditional Machine Translation (MT) systems are designed to translate documents. In this paper we describe an MT system that translates the closed captions that accompany most North American television broadcasts. This domain has two identifying characteristics. First, the captions themselves have properties quite different from the type of textual input that many MT systems have been designed for. This is due to the fact that captions generally represent speech and hence contain many of the phenomena that characterize spoken language. Second, the operational characteristics of the closed-caption domain are also quite distinctive. Unlike most other translation domains, the translated captions are only one of several sources of information that are available to the user. In addition, the user has limited time to comprehend the translation since captions only appear on the screen for a few seconds. In this paper, we look at some of the theoretical and implementational challenges that these characteristics pose for MT. We present a fully automatic large-scale multilingual MT system, ALTo. Our approach is based on Whitelock's Shake and Bake MT paradigm, which relies heavily on lexical resources. The system currently provides wide-coverage translation from English to Spanish. In addition to discussing the design of the system, we also address the evaluation issues that are associated with this domain and report on our current performance.     

RNN编码器-解码器在维汉机器翻译中的应用

将RNN编码器-解码器作为传统的基于短语的PSMT系统的一部分,在传统统计机器翻译基础上,集成RNN解码器-编码器,兼容PSMT创建了新联合模型（RNN+PSMT）。新的模型不仅在维-汉、汉-英机器翻译的应用中取得了成效,而且能够捕捉到语言的规律,使得机器翻译中的一个重要评价指标的BLEU值得到了显著提高。实验结果表明,系统的整体性能超过了传统统计机器翻译。     

Controlled Translation in an Example-based Environment: What do Automatic Evaluation Metrics Tell Us?

This paper presents an extended, harmonised account of our previous work on integrating controlled language data in an Example-based Machine Translation system. Gough and Way in MT Summit pp. 133–140 (2003) focused on controlling the output text in a novel manner, while Gough and Way (9th Workshop of the EAMT, (2004a), pp. 73–81) sought to constrain the input strings according to controlled language specifications. Our original sub-sentential alignment algorithm could deal only with 1:1 matches, but subsequent refinements enabled n:m alignments to be captured. A direct consequence was that we were able to populate the system’s databases with more than six times as many potentially useful fragments. Together with two simple novel improvements – correcting a small number of mistranslations in the lexicon, and allowing multiple translations in the lexicon – translation quality improves considerably. We provide detailed automatic and human evaluations of a number of experiments carried out to test the quality of the system. We observe that our system outperforms the rule-based on-line system Logomedia on a range of automatic evaluation metrics, and that the ‘best’ translation candidate is consistently highly ranked by our system. Finally, we note in a number of tests that the BLEU metric gives objectively different results than other automatic evaluation metrics and a manual evaluation. Despite these conflicting results, we observe a preference for controlling the source data rather than the target translations.     

交互式机器翻译综述

随着深度学习的发展和成熟，神经机器翻译的质量也越来越高，然而仍不完美，为了达到可接受的翻译效果，需要人工进行后期编辑。交互式机器翻译（IMT）是这种串行工作的一个替代，即在翻译过程中进行人工互动，由用户对翻译系统产生的候选翻译进行验证，并且，如有必要，由用户提供新的输入，系统根据用户当前的反馈生成新的候选译文，如此往复，直到产生一个使用户满意的输出。首先，介绍了IMT的基本概念以及当前的研究进展；然后，分类对一些常用方法和前沿工作加以介绍，并简述每个工作的背景和创新之处；最后，探讨了IMT的发展趋势和研究难点。     

Selective sampling techniques for feedback-based data retrieval

As many databases have been brought online, data retrieval??finding relevant data from large databases??has become a nontrivial task. A feedback-based data retrieval system was proposed to provide user with an intuitive way for expressing their preferences in queries. The system iteratively receives a partial ordering on a sample of data from the user, learns a ranking function, and returns highly ranked results according to the function. An important issue in such retrieval systems is minimizing the number of iterations or the amount of feedback to learn an accurate ranking function. This paper proposes selective sampling (or active learning) techniques for RankSVM that can be used in the retrieval systems. The proposed techniques minimizes the amount of user interaction to learn an accurate ranking function thus facilitates users formulating a preference query in the data retrieval system.     

一种改进的维吾尔语句子相似度计算方法

在基于实例的维吾尔语汉语机器翻译系统中维吾尔语相似度计算起重要作用。维吾尔语的黏着性特性要求对单词进行词干提取。本文提出的方法结合简单的句子结构相似度计算方法,通过对单词词干提取进行句子相似度计算。小规模实验结果比较接近人工评价的句子相似度。     

Cunei: open-source machine translation with relevance-based models of each translation instance

The Cunei machine translation platform is an open-source system for data-driven machine translation. Our platform is a synthesis of the traditional example-based MT (EBMT) and statistical MT (SMT) paradigms. What makes Cunei unique is that it measures the relevance of each translation instance with a distance function. This distance function, represented as a log-linear model, operates over one translation instance at a time and enables us to score the translation instance relative to the specified input and/or the current target hypothesis. We describe how our system, Cunei, scores features individually for each translation instance and how it efficiently performs parameter tuning over the entire feature space. We also compare Cunei with three other open-source MT systems (Moses, CMU-EBMT, and Marclator). In our experiments involving Korean–English and Czech–English translation Cunei clearly outperforms the traditional EBMT and SMT systems.     

韩国语句子结构相似度计算方法研究

句子相似度计算是信息处理领域一项基础技术,在基于实例的机器翻译中直接影响译文质量。该文以韩国语句子为研究对象,结合韩国语的句子特点提出了一种句子结构相似度的计算方法。该方法通过先提取句子的骨架结构,然后结合韩国语的句法特点制定标记转换规则,最后用转换后的句子结构与实例库中句子匹配得到与之相似的句子,得出两个句子间的结构相似度,并且通过实验验证了该方法的可行性,提高了相似度计算效果。     

The Long-Term Forecast for Weather Bulletin Translation

Machine Translation (MT) is the focus of extensive scientific investigations driven by regular evaluation campaigns, but which are mostly oriented towards a somewhat particular task: translating news articles into English. In this paper, we investigate how well current MT approaches deal with a real-world task. We have rationally reconstructed one of the only MT systems in daily use which produces high-quality translation: the Météo system. We show how a combination of a sentence-based memory approach, a phrase-based statistical engine and a neural-network rescorer can give results comparable to those of the current system. We also explore another possible prospect for MT technology: the translation of weather alerts, which are currently being translated manually by translators at the Canadian Translation Bureau.     

基于CFN的相似度计算在实例机器翻译中的应用

在信息检索,文本挖掘以及基于实例的机器翻译中,相似度计算都是一个关键问题.在实例机器翻译中,相似度计算一般是基于字符、词的匹配以及向量空间模型,但基于句子语义结构的相似度研究还不多见.借助了汉语框架语义网(Chinese FrameNet,简称CFN)的场景语义描述优势,提出了一种新的面向EBMT进行实例相似度计算的方...