A conjoint analysis framework for evaluating user preferences in machine translation

Despite much research on machine translation (MT) evaluation, there is surprisingly little work that directly measures users’ intuitive or emotional preferences regarding different types of MT errors. However, the elicitation and modeling of user preferences is an important prerequisite for research on user adaptation and customization of MT engines. In this paper we explore the use of conjoint analysis as a formal quantitative framework to assess users’ relative preferences for different types of translation errors. We apply our approach to the analysis of MT output from translating public health documents from English into Spanish. Our results indicate that word order errors are clearly the most dispreferred error type, followed by word sense, morphological, and function word errors. The conjoint analysis-based model is able to predict user preferences more accurately than a baseline model that chooses the translation with the fewest errors overall. Additionally we analyze the effect of using a crowd-sourced respondent population versus a sample of domain experts and observe that main preference effects are remarkably stable across the two samples.     

基于双语短语约束的交互式机器翻译方法

交互式机器翻译(Interactive Machine Translation,IMT)是一种通过机器翻译系统与译员之间的相互作用指导计算机解码并改善输出译文质量的技术。目前主流的IMT方法使用译员确定的前缀作为唯一约束指导解码,交互方式受限,交互效率低。该文从交互方式和解码算法两个方面对IMT方法进行改进。在交互方式方面,允许译员译前从短语译项列表中为源语言短语选择正确译项。该文还提出了基于短语表的多样性排序算法,来提高短语候选译项的多样性,并根据译员的翻译认知过程设计交互界面,改善译员在翻译过程中的用户体验。在解码算法方面,将双语短语与前缀一同作为约束参与指导解码过程,提高翻译假设评价和过滤的准确性。在LDC汉英平行语料上进行了人工评测,实验结果表明该方法较传统的IMT方法能够减轻译员的认知负担,减少翻译时间,提升翻译效率。     

Novel algorithm machine translation for language translation tool

Fuzzy matching techniques are the presently used methods in translating the words. Neural machine translation and statistical machine translation are the methods used in MT. In machine translator tool, the strategy employed for translation needs to handle large amount of datasets and therefore the performance in retrieving correct matching output can be affected. In order to improve the matching score of MT, the advanced techniques can be presented by modifying the existing fuzzy based translator and neural machine translator. The conventional process of modifying architectures and encoding schemes are tedious process. Similarly, the preprocessing of datasets also involves more time consumption and memory utilization. In this article, a new spider web based searching enhanced translation is presented to be employed with the neural machine translator. The proposed scheme enables deep searching of available dataset to detect the accurate matching result. In addition, the quality of translation is improved by presenting an optimal selection scheme for using the sentence matches in source augmentation. The matches retrieved using various matching scores are applied to an optimization algorithm. The source augmentation using optimal retrieved matches increases the translation quality. Further, the selection of optimal match combination helps to reduce time requirement, since it is not necessary to test all retrieved matches in finding target sentence. The performance of translation is validated by measuring the quality of translation using BLEU and METEOR scores. These two scores can be achieved for the TA-EN language pairs in different configurations of about 92% and 86%, correspondingly. The results are evaluated and compared with other available NMT methods to validate the work.     

Online discriminative learning for machine translation with binary-valued feedback

Viewing machine translation (MT) as a structured classification problem has provided a gateway for a host of structured prediction techniques to enter the field. In particular, large-margin methods for discriminative training of feature weights, such as the structured perceptron or MIRA, have started to match or exceed the performance of existing methods such as MERT. One issue with these problems in general is the difficulty in obtaining fully structured labels, e.g. in MT, obtaining reference translations or parallel sentence corpora for arbitrary language pairs. Another issue, more specific to the translation domain, is the difficulty in online training and updating of MT systems, since existing methods often require bilingual knowledge to correct translation outputs online. The problem is an important one, especially with the usage of MT in the mobile domain: in the process of translating user inputs, these systems can also receive feedback from the user on the quality of the translations produced. We propose a solution to these two problems, by demonstrating a principled way to incorporate binary-labeled feedback (i.e. feedback on whether a translation hypothesis is a “good” or understandable one or not), a form of supervision that can be easily integrated in an online and monolingual manner, into an MT framework. Experimental results on Chinese–English and Arabic–English corpora for both sparse and dense feature sets show marked improvements by incorporating binary feedback on unseen test data, with gains in some cases exceeding 5.5 BLEU points. Experiments with human evaluators providing feedback present reasonable correspondence with the larger-scale, synthetic experiments and underline the relative ease by which binary feedback for translation hypotheses can be collected, in comparison to parallel data.     

基于分层语块分析的统计翻译研究

本文描述了一个基于分层语块分析的统计翻译模型。该模型在形式上不仅符合同步上下文无关文法,而且融合了基于条件随机场的英文语块分析知识,因此基于分层语块分析的统计翻译模型做到了将句法翻译模型和短语翻译模型有效地结合。该系统的解码算法改进了线图分析的CKY算法,融入了线性的N-gram语言模型。目前,本文主要针对中文－英文的口语翻译进行了一系列实验,并以国际口语评测IWSLT(International Workshop on Spoken Language Translation)为标准,在2005年的评测测试集上,BLEU和NIST得分均比统计短语翻译系统有所提高。     

受限汉语与汉英机器翻译系统

随着进入全球化的经济时代,对机器翻译的市场需求也正在急剧增长.传统的机器翻译依然只能作为一种辅助翻译工具.如果坚持对汉语的真实文本进行机器翻译路线,则不可能使汉英机器翻译质量取得实质性的突破.从计算机技术在自然语言信息处理领域的实际能力出发,开发面向受限汉语的汉英机器翻译系统,是机器翻译技术未来发展的基本方向.     

一种在线递增式语言模型自适应方法

本文针对传统统计语言模型的离线自适应方法,提出了一种在线实时的递增式自适应方法。该自适应方法需要解决几个问题。第一是要设计一种语言模型结构以适应在线的自适应;第二是如何利用在线收集到的语料对语言模型进行实时的参数修改;在我们设计的中文音转字平台中,将语言模型分成两个部分,分别是通用模型和用户模型。对于通用模型,采用高效的存储结构结合参数预取技术,提高了模型的速度;对于用户模型,使用动态的加权方法结合MAP 动态调整参数。本文所做的实验证明使用该方法能较大程度的降低中文音转字的错误率。     

Tabular-expression-based method for constructing metamorphic relations

Metamorphic testing (MT) is proposed to overcome the oracle problem in software testing, and metamorphic relations (MRs) are the core of MT. There is a lack of guidelines for constructing effective MRs, and it is difficult to reuse MRs mainly because most MRs are closely related to the domain knowledge. In this article, we propose a method for constructing MRs from specifications in tabular expression format. Our method constructs MRs according to the characteristics of tabular expressions, especially the relationships between the header grids and the main grid, namely, our method is domain-independent and the construction process is simplified. In addition, the derived MRs can be applied to specifications with the same tabular expression structure. For specifications with different tabular expression structures, MRs can still be used after slight adjustments. To evaluate the performance of our method in practice, we apply the method to five applications. The experimental results demonstrate that our method is effective for a program with the oracle problem, and that it is applicable to tabular expressions in various formats. Compared with representative testing methods, our method identifies errors that are not detected by the compared methods. Hence, our method and existing methods can complement each other. The MR proposed in this article outperforms MRs constructed based on program properties.     

基于句对质量和覆盖度的统计机器翻译训练语料选取

该文研究的目的是在待翻译文本未知的情况下,从已有的大规模平行语料中选取一个高质量的子集作为统计机器翻译系统的训练语料,以降低训练和解码代价。该文综合覆盖度和句对翻译质量两方面因素,提出一种从已有平行语料中获取高质量小规模训练子集的方法。在CWMT2008汉英翻译任务上的实验结果表明,利用本文的方法能够从现有大规模语料中选取高质量的子集,在减少80%训练语料的情况下达到与Baseline系统(使用全部训练语料)相当的翻译性能(BLEU值)。     

Using target-language information to train part-of-speech taggers for machine translation

Although corpus-based approaches to machine translation (MT) are growing in interest, they are not applicable when the translation involves less-resourced language pairs for which there are no parallel corpora available; in those cases, the rule-based approach is the only applicable solution. Most rule-based MT systems make use of part-of-speech (PoS) taggers to solve the PoS ambiguities in the source-language texts to translate; those MT systems require accurate PoS taggers to produce reliable translations in the target language (TL). The standard statistical approach to PoS ambiguity resolution (or tagging) uses hidden Markov models (HMM) trained in a supervised way from hand-tagged corpora, an expensive resource not always available, or in an unsupervised way through the Baum-Welch expectation-maximization algorithm; both methods use information only from the language being tagged. However, when tagging is considered as an intermediate task for the translation procedure, that is, when the PoS tagger is to be embedded as a module within an MT system, information from the TL can be (unsupervisedly) used in the training phase to increase the translation quality of the whole MT system. This paper presents a method to train HMM-based PoS taggers to be used in MT; the new method uses not only information from the source language (SL), as general-purpose methods do, but also information from the TL and from the remaining modules of the MT system in which the PoS tagger is to be embedded. We find that the translation quality of the MT system embedding a PoS tagger trained in an unsupervised manner through this new method is clearly better than that of the same MT system embedding a PoS tagger trained through the Baum-Welch algorithm, and comparable to that obtained by embedding a PoS tagger trained in a supervised way from hand-tagged corpora.     

Bilingual LSA-based adaptation for statistical machine translation

We propose a novel approach to cross-lingual language model and translation lexicon adaptation for statistical machine translation (SMT) based on bilingual latent semantic analysis. Bilingual LSA enables latent topic distributions to be efficiently transferred across languages by enforcing a one-to-one topic correspondence during training. Using the proposed bilingual LSA framework, model adaptation can be performed by, first, inferring the topic posterior distribution of the source text and then applying the inferred distribution to an n-gram language model of the target language and translation lexicon via marginal adaptation. The background phrase table is enhanced with the additional phrase scores computed using the adapted translation lexicon. The proposed framework also features rapid bootstrapping of LSA models for new languages based on a source LSA model of another language. Our approach is evaluated on the Chinese–English MT06 test set using the medium-scale SMT system and the GALE SMT system measured in BLEU and NIST scores. Improvement in both scores is observed on both systems when the adapted language model and the adapted translation lexicon are applied individually. When the adapted language model and the adapted translation lexicon are applied simultaneously, the gain is additive. At the 95% confidence interval of the unadapted baseline system, the gain in both scores is statistically significant using the medium-scale SMT system, while the gain in the NIST score is statistically significant using the GALE SMT system.     

The NIST 2008 Metrics for machine translation challenge—overview, methodology, metrics, and results

This paper discusses the evaluation of automated metrics developed for the purpose of evaluating machine translation (MT) technology. A general discussion of the usefulness of automated metrics is offered. The NIST MetricsMATR evaluation of MT metrology is described, including its objectives, protocols, participants, and test data. The methodology employed to evaluate the submitted metrics is reviewed. A summary is provided for the general classes of evaluated metrics. Overall results of this evaluation are presented, primarily by means of correlation statistics, showing the degree of agreement between the automated metric scores and the scores of human judgments. Metrics are analyzed at the sentence, document, and system level with results conditioned by various properties of the test data. This paper concludes with some perspective on the improvements that should be incorporated into future evaluations of metrics for MT evaluation.     

Example-based machine translation based on tree–string correspondence and statistical generation

This paper describes an example-based machine translation (EBMT) method based on tree–string correspondence (TSC) and statistical generation. In this method, the translation example is represented as a TSC, which is a triple consisting of a parse tree in the source language, a string in the target language, and the correspondence between the leaf node of the source-language tree and the substring of the target-language string. For an input sentence to be translated, it is first parsed into a tree. Then the TSC forest which best matches the input tree is searched for. Finally the translation is generated using a statistical generation model to combine the target-language strings of the TSCs. The generation model consists of three features: the semantic similarity between the tree in the TSC and the input tree, the translation probability of translating the source word into the target word, and the language-model probability for the target-language string. Based on the above method, we build an English-to-Chinese MT system. Experimental results indicate that the performance of our system is comparable with phrase-based statistical MT systems.     

一种混合策略的专利机器翻译系统研究

针对专利文本翻译中的复杂语句,提出了一种基于混合策略的方法,融合语义分析技术和基于规则的翻译技术,来提高专利翻译的效果。利用语义分析技术,重点解决句子中心动词识别和句子中有嵌套结构存在的名称短语的分析,把语义分析结果输入到基于规则的翻译系统中,用以改善翻译的效果。测试结果表明,融合后的翻译系统,BLEU值提高了9.8%。该方法已经集成到了国家知识产权局的在线汉英机器翻译系统中,有效地提高了专利翻译的效果和翻译效率。     

Reassessing the proper place of man and machine in translation: a pre-translation scenario

Traditionally, human–machine interaction to reach an improved machine translation (MT) output takes place ex-post and consists of correcting this output. In this work, we investigate other modes of intervention in the MT process. We propose a Pre-Edition protocol that involves: (a) the detection of MT translation difficulties; (b) the resolution of those difficulties by a human translator, who provides their translations (pre-translation); and (c) the integration of the obtained information prior to the automatic translation. This approach can meet individual interaction preferences of certain translators and can be particularly useful for production environments, where more control over output quality is needed. Early resolution of translation difficulties can prevent downstream errors, thus improving the final translation quality “for free”. We show that translation difficulty can be reliably predicted for English for various source units. We demonstrate that the pre-translation information can be successfully exploited by an MT system and that the indirect effects are genuine, accounting for around 16% of the total improvement. We also provide a study of the human effort involved in the resolution process.     

Cunei: open-source machine translation with relevance-based models of each translation instance

The Cunei machine translation platform is an open-source system for data-driven machine translation. Our platform is a synthesis of the traditional example-based MT (EBMT) and statistical MT (SMT) paradigms. What makes Cunei unique is that it measures the relevance of each translation instance with a distance function. This distance function, represented as a log-linear model, operates over one translation instance at a time and enables us to score the translation instance relative to the specified input and/or the current target hypothesis. We describe how our system, Cunei, scores features individually for each translation instance and how it efficiently performs parameter tuning over the entire feature space. We also compare Cunei with three other open-source MT systems (Moses, CMU-EBMT, and Marclator). In our experiments involving Korean–English and Czech–English translation Cunei clearly outperforms the traditional EBMT and SMT systems.     

多策略汉日机器翻译系统中的核心技术研究

多策略的机器翻译是当今机器翻译系统的一个发展方向。该文论述了一个多策略的汉日机器翻译系统中各翻译核心子系统所使用的核心技术和算法,其中包含了使用词法分析、句法分析和语义角色标注的汉语分析子系统、利用双重索引技术的基于翻译记忆技术的机器翻译子系统、以句法树片段为模板的基于实例模式的机器翻译子系统以及综合了配价模式和断段分析的机器翻译子系统。翻译记忆子系统的测试结果表明其具有高效的特性;实例模式子系统在1 559个句子的封闭测试中达到99%的准确率,在1 500个句子的开放测试中达到85%的准确率;配价模式子系统在3 059个句子的测试中达到了89%的准确率。     

Weight adaptation and oscillatory correlation for imagesegmentation

We propose a method for image segmentation based on a neural oscillator network. Unlike previous methods, weight adaptation is adopted during segmentation to remove noise and preserve significant discontinuities in an image. Moreover, a logarithmic grouping rule is proposed to facilitate grouping of oscillators representing pixels with coherent properties. We show that weight adaptation plays the roles of noise removal and feature preservation. In particular, our weight adaptation scheme is insensitive to termination time and the resulting dynamic weights in a wide range of iterations lead to the same segmentation results. A computer algorithm derived from oscillatory dynamics is applied to synthetic and real images, and simulation results show that the algorithm yields favorable segmentation results in comparison with other recent algorithms. In addition, the weight adaptation scheme can be directly transformed to a novel feature-preserving smoothing procedure. We also demonstrate that our nonlinear smoothing algorithm achieves good results for various kinds of images.     

基于WordNet词义消歧的系统融合

最近混淆网络在融合多个机器翻译结果中展示很好的性能. 然而为了克服在不同的翻译系统中不同的词序, 假设对齐在混淆网络的构建上仍然是一个重要的问题. 但以往的对齐方法都没有考虑到语义信息. 本文为了更好地改进系统融合的性能, 提出了用词义消歧(Word sense disambiguation, WSD)来指导混淆网络中的对齐. 同时骨架翻译的选择也是通过计算句子间的相似度来获得的, 句子的相似性计算使用了二分图的最大匹配算法. 为了使得基于WordNet词义消歧方法融入到系统中, 本文将翻译错误率(Translation error rate, TER)算法进行了改进, 实验结果显示本方法的性能好于经典的TER算法的性能.