“细粒度英汉机器翻译错误分析语料库”的构建与思考

机器翻译错误分析旨在找出机器译文中存在的错误,包括错误类型、错误分布等,它在机器翻译研究和应用中发挥着重要作用。该文将人工译后编辑与错误分析结合起来,对译后编辑操作进行错误标注,采用自动标注和人工标注相结合的方法,构建了一个细粒度英汉机器翻译错误分析语料库,其中每一个标注样本包括源语言句子、机器译文、人工参考译文、译后编辑译文、词错误率和错误类型标注;标注的错误类型包括增词、漏词、错词、词序错误、未译和命名实体翻译错误等。标注的一致性检验表明了标注的有效性;对标注语料的统计分析结果能有效地指导机器翻译系统的开发和人工译员的后编辑。     

基于成分共享的英汉小句对齐语料库标注体系研究

英汉小句对齐语料库服务于英语和汉语小句的语法结构对应关系研究和应用,对于语言理论和语言翻译(包括人的翻译和机器翻译)有重要意义。前人的语法理论和相关语料库的工作对于小句复合体和小句的界定缺乏充分研究,在理论上有缺陷,难以支持自然语言处理的应用。该文首先为英汉小句对齐语料库的建设做理论准备。从近年提出的汉语小句复合体的理论出发,该文界定了成分共享的概念,基于话头共享和引语共享来界定英语的小句和小句复合体,使小句和小句复合体具有功能的完整性和单一性。在此基础上,该文设计了英汉小句对齐的标注体系,包括英语NT小句标注和汉语译文生成及组合。语料库的标注表明,在小句复合体层面上英汉翻译涉及到的结构变换,其部件可以限制为英语小句和话头、话体,无须涉及话头和话体内部的结构。基于这些工作的英汉小句对齐语料库为语言本体研究和英汉语言对比、英汉机器翻译等应用提供了结构化的标注样本。     

跨语言命名实体翻译对抽取的研究综述

跨语言命名实体对于机器翻译、跨语言信息抽取都具有重要意义,从命名实体的音译、基于平行/可比语料库的跨语言命名实体对齐、基于网络挖掘的跨语言命名实体对翻译抽取3个方面对跨语言命名实体翻译对抽取的研究现状进行了总结。音译是跨语言命名实体翻译对抽取的重点内容之一,基于深度学习的音译模型将是今后的研究重点。目前,跨语言平行/可比语料库的获取和标注直接影响基于语料库的跨语言命名实体对齐的深入研究。基于信息检索和维基百科的跨语言命名实体翻译对抽取研究将是跨语言命名实体翻译对抽取研究的趋势。     

Exploring the <Emphasis Type="SmallCaps">sawa</Emphasis> corpus: collection and deployment of a parallel corpus English—Swahili

Research in machine translation and corpus annotation has greatly benefited from the increasing availability of word-aligned parallel corpora. This paper presents ongoing research on the development and application of the sawa corpus, a two-million-word parallel corpus English—Swahili. We describe the data collection phase and zero in on the difficulties of finding appropriate and easily accessible data for this language pair. In the data annotation phase, the corpus was semi-automatically sentence and word-aligned and morphosyntactic information was added to both the English and Swahili portion of the corpus. The annotated parallel corpus allows us to investigate two possible uses. We describe experiments with the projection of part-of-speech tagging annotation from English onto Swahili, as well as the development of a basic statistical machine translation system for this language pair, using the parallel corpus and a consolidated database of existing English—Swahili translation dictionaries. We particularly focus on the difficulties of translating English into the morphologically more complex Bantu language of Swahili.     

濒危语言口语语料库的研究与构建——以吕苏语为范例

濒危语言口语语料库建立的目的是系统地保存近乎消失的濒危语言,留存濒危语言的生命力与地方文化,并且能够对其进行学习与研究。濒危语言口语语料库保存的内容主要包括原始声音文件、国际音标标注、汉语对译标注以及汉语翻译标注。以濒危语言吕苏语为范例,深入、全面、系统地研究与建立濒危语言口语语料库,并对标注语料实现了自动分词与关键词提取的功能,为后续建立通用濒危语言语料库提供了一个范例。     

基于机器学习的平行语料库应用效果评价方法

针对传统翻译系统在时态翻译中不准确的问题,结合当前的机器学习算法,提出一种基于DBN的平行语料库时态翻译方法。为实现该方法,首先对时态标注模型和DBN基本理论进行介绍,并提出汉英语句时态翻译的思路;而在进行DBN平行语料库特征提取的过程中,采用自动时态标注算法对时态进行标注,并对得到的数据进行时态树编码;然后以编码数据作为输入,运用DBN网络对时态进行训练预测,得到中文语句中可能的时态;最后通过Transformer翻译器对语句进行翻译,得到对应的翻译时态句子;最后结合LDC等平行语料库中的翻译句子,采用上述方法进行翻译,结果表明提出的时态翻译方法无论是在准确率,还是在召回率等方面,都有很大的优势,说明提出的机器学习算法在平行语料库的翻译中更具有准确性。     

A French clinical corpus with comprehensive semantic annotations: development of the Medical Entity and Relation LIMSI annOtated Text corpus (MERLOT)

Quality annotated resources are essential for Natural Language Processing. The objective of this work is to present a corpus of clinical narratives in French annotated for linguistic, semantic and structural information, aimed at clinical information extraction. Six annotators contributed to the corpus annotation, using a comprehensive annotation scheme covering 21 entities, 11 attributes and 37 relations. All annotators trained on a small, common portion of the corpus before proceeding independently. An automatic tool was used to produce entity and attribute pre-annotations. About a tenth of the corpus was doubly annotated and annotation differences were resolved in consensus meetings. To ensure annotation consistency throughout the corpus, we devised harmonization tools to automatically identify annotation differences to be addressed to improve the overall corpus quality. The annotation project spanned over 24 months and resulted in a corpus comprising 500 documents (148,476 tokens) annotated with 44,740 entities and 26,478 relations. The average inter-annotator agreement is 0.793 F-measure for entities and 0.789 for relations. The performance of the pre-annotation tool for entities reached 0.814 F-measure when sufficient training data was available. The performance of our entity pre-annotation tool shows the value of the corpus to build and evaluate information extraction methods. In addition, we introduced harmonization methods that further improved the quality of annotations in the corpus.     

DO WE AGREE? MEASURING AGREEMENT ON THE HUMAN JUDGMENTS IN EMOTION ANNOTATION OF NEWS SENTENCES

An emotional text may be judged to belong to multiple emotion categories because it may evoke different emotions with varying degrees of intensity. For emotion analysis of text in a supervised manner, it is required to annotate text corpus with emotion categories. Because emotion is a very subjective entity, producing reliable annotation is of prime requirement for developing a robust emotion analysis model, so it is wise to have the data set annotated by multiple human judges and generate an aggregated data set provided that the emotional responses provided by different annotators over the data set exhibit substantial agreement. In reality, multiple emotional responses for an emotional text are common. So, the data set is a multilabel one where a single data item may belong to more than one category simultaneously. This article presents a new agreement measure to compute interannotator reliability in multilabel annotation. The new reliability coefficient has been applied to measure the quality of an emotion text corpus. The procedure for generating aggregated data and some corpus cleaning techniques are also discussed.     

基于机器翻译的跨语言关系抽取

训练语料库的规模对基于机器学习的命名实体间语义关系抽取具有重要的作用,而语料库的人工标注需要花费大量的时间和人力。该文提出了使用机器翻译的方法将源语言的关系实例转换成目标语言的关系实例,并通过实体对齐策略将它们加入到目标语言的训练集中,从而使资源丰富的源语言帮助欠资源的目标语言进行语义关系抽取。在ACE2005中英文语料库上的关系抽取实验表明,无论是将中文翻译成英文,还是将英文翻译成中文,都对另一种语言的关系抽取具有帮助作用。特别是当目标语言的训练语料库规模较小时,这种帮助就尤其显著。     

中文核心框架元素标注规则的获取和实验——以认知语义领域为例

计算机语义分析是当前制约中文信息检索、信息抽取与机器翻译等应用技术发展的瓶颈问题,而语义角色标注是语义分析的一个主要任务.即以框架语义理论为基础,以汉语框架语义知识库的核心框架元素为语义角色标注的范畴知识体系,以认知语义领域为研究实例,以真实语料为依据,根据短语类型、句法功能以及短语内、外部其他句法语义特征,获取中文语...     

基于邮件过滤的中文邮件语料库构建

首先分析了现阶段邮件过滤的主要技术和邮件语料库建设的现状,并提出了建设中文邮件语料库的相关问题,建议在邮件建设过程中保留邮件信头信息、不排斥邮件副本.然后给出了邮件语料库系统的实现框架,分为邮件源代码的解析与预处理、邮件的初次标注、词分类和邮件的二次标注四个步骤,并通过提供一个管理工具来管理邮件语料.最后,介绍了目前已经建设的一个邮件语料库的情况.     

Iarg-AnCora: Spanish corpus annotated with implicit arguments

This article presents the Spanish Iarg-AnCora corpus (400 k-words, 13,883 sentences) annotated with the implicit arguments of deverbal nominalizations (18,397 occurrences). We describe the methodology used to create it, focusing on the annotation scheme and criteria adopted. The corpus was manually annotated and an interannotator agreement test was conducted (81 % observed agreement) in order to ensure the reliability of the final resource. The annotation of implicit arguments results in an important gain in argument and thematic role coverage (128 % on average). It is the first corpus annotated with implicit arguments for the Spanish language with a wide coverage that is freely available. This corpus can subsequently be used by machine learning-based semantic role labeling systems, and for the linguistic analysis of implicit arguments grounded on real data. Semantic analyzers are essential components of current language technology applications, which need to obtain a deeper understanding of the text in order to make inferences at the highest level to obtain qualitative improvements in the results.     

The Bible as a Parallel Corpus: Annotating the ‘Book of 2000 Tongues’

We report on a project to annotate biblical texts in order to create an aligned multilingual Bible corpus for linguistic research, particularly computational linguistics, including automatically creating and evaluating translation lexicons and semantically tagged texts. The output of this project will enable researchers to take advantage of parallel translations across a wider number of languages than previously available, providing, with relatively little effort, a corpus that contains careful translations and reliable alignment at the near-sentence level. We discuss the nature of the text, our annotation process, preliminary and planned uses for the corpus, and relevant aspects of the Corpus Encoding Standard (CES) with respect to this corpus. We also present a quantitative comparison with dictionary and corpus resources for modern-day English, confirming the relevance of this corpus for research on present day language.     

基于变分信息瓶颈的半监督神经机器翻译

变分方法是机器翻译领域的有效方法, 其性能较依赖于数据量规模. 然而在低资源环境下, 平行语料资源匮乏, 不能满足变分方法对数据量的需求, 因此导致基于变分的模型翻译效果并不理想. 针对该问题, 本文提出基于变分信息瓶颈的半监督神经机器翻译方法, 所提方法的具体思路为: 首先在小规模平行语料的基础上, 通过引入跨层注意力机制充分利用神经网络各层特征信息, 训练得到基础翻译模型; 随后, 利用基础翻译模型, 使用回译方法从单语语料生成含噪声的大规模伪平行语料, 对两种平行语料进行合并形成组合语料, 使其在规模上能够满足变分方法对数据量的需求; 最后, 为了减少组合语料中的噪声, 利用变分信息瓶颈方法在源与目标之间添加中间表征, 通过训练使该表征具有放行重要信息、阻止非重要信息流过的能力, 从而达到去除噪声的效果. 多个数据集上的实验结果表明, 本文所提方法能够显著地提高译文质量, 是一种适用于低资源场景的半监督神经机器翻译方法.     

一种命名实体翻译等价对的抽取方法

有关命名实体的翻译等价对在多语言处理中有着非常重要的意义。在过去的几年里,双语字典查找,音译模型等方法先后被提出。另一种极具价值的方法是从平行语料库中自动抽取有关命名实体的翻译等价对,现有的方法要求预先对双语语料库的两种语言文本进行命名实体标注。提出了一种只要求对语料库中源语言进行命名实体标注,目标语言不需标注,然后利用训练得到的HMM词对齐结果来抽取有关命名实体翻译等价对的方法。在实验中,把中文作为源语言,英文作为目标语言。实验结果表明用该方法,即使在对齐模型只是部分准确的情况下,也得到了较高正确率的命名实体翻译等价对。     

傣文自动分词系统的设计与实现

傣文自动分词是傣文信息处理中的基础工作,是后续进行傣文输入法开发、傣文自动机器翻译系统开发、傣文文本信息抽取等傣文信息处理的基础,受限于傣语语料库技术,傣文自然语言处理技术较为薄弱。本文首先对傣文特点进行了分析, 并在此基础上构建了傣文语料库,同时将中文分词方法应用到傣文中,结合傣文自身的特点,设计了一个基于音节序列标注的傣文分词系统,经过实验,该分词系统达到了95.58%的综合评价值。     

汉英篇章结构平行语料库的对齐标注研究

篇章结构平行语料库是对具有对译关系的双语文本标注了平行篇章结构信息的语料库。对齐标注是汉英篇章结构平行语料库的核心理论基础。该文提出“结构对齐,关系对齐”的对齐标注策略,应用于切分对齐、层次结构对齐、关系对齐、中心对齐等环节,实现了对齐和标注并行、单位对齐和结构对齐共进的平行语料库工作模式。本策略辅之以相应标注平台和工作程序以及相应难点解决方案,被证明是一种高效的篇章结构平行语料库工作方式。     

Evaluating and automating the annotation of a learner corpus

The paper describes a corpus of texts produced by non-native speakers of Czech. We discuss its annotation scheme, consisting of three interlinked tiers, designed to handle a wide range of error types present in the input. Each tier corrects different types of errors; links between the tiers allow capturing errors in word order and complex discontinuous expressions. Errors are not only corrected, but also classified. The annotation scheme is tested on a data set including approx. 175,000 words with fair inter-annotator agreement results. We also explore the possibility of applying automated linguistic annotation tools (taggers, spell checkers and grammar checkers) to the learner text to support or even substitute manual annotation.     

解决多音字字-音转换的一种统计学习方法

字-音转换是语音合成系统中的一个重要模块,其中多音词和以单字词形式存在的多音字读音的确定一直是个没有很好解决的问题。本文通过对大量标注有正确拼音的语料的统计,指出着重解决41个重点多音字和22个重点多音词就可基本解决字-音转换的问题。本文采用基于扩展的随机复杂度的随机决策列表方法自动提取多音字(词)的读音规则,将字-音转换的错误率由8.8‰降低到4.4‰。规则的训练和测试的材料的标注是一个耗费人力和时间的工作,而训练材料的数量和质量又直接影响最终的结果。本文提出一种半自动的语料标注流程,可以节省将近一半的人工和时间。