面向多领域多来源文本的汉语依存句法树库构建

为了支持汉语句法分析研究,目前句法分析领域已经标注了多个汉语依存句法树库。然而,已有树库主要针对较规范文本,而对各种网络文本如博客、微博、微信等考虑较少。为此,该文基于近期研制的标注规范及可视化在线标注系统,开展了大规模数据标注。聘请了15名兼职标注者,并采用严格的标注流程保证标注质量,目前,已经标注了约3万句的汉语依存句法树库,其中包含约1万句淘宝头条文本。该文重点介绍了数据选取、标注流程等问题,并详细分析了标注准确率、一致性和标注数据的分布情况。未来将继续对多领域多来源文本进行标注,扩大树库规模,并以合适的方式公开相应的标注数据。     

汉语学习者依存句法树库构建

汉语学习者依存句法树库为非母语者语料提供依存句法分析,对第二语言教学与研究,以及面向第二语言的句法分析、语法改错等相关研究有重要意义.然而,现有的汉语学习者依存句法树库数量较少,且在标注方面仍存在一些问题.为此,该文提出一个依存句法标注规范,搭建在线标注平台,并开展汉语学习者依存句法标注.该文重点介绍了数据选取、标注流...     

多领域中文依存树库构建与影响统计句法分析因素之分析

为提升依存分析并分析影响其精度的相关因素,该文构建了大规模中文通用依存树库和中等规模领域依存树库。基于这一系列树库,通过句法分析实验考察质量、规模、领域差异等因素对中文依存分析的影响,实验结果表明: (1)树库规模和质量均与句法分析精度成正相关关系,质量应先于规模因素被优先考虑;(2)通用树库和领域树库之间的差异程度与前者对后者的替代性成相关关系;(3)两种树库混合使用的效果同样与领域差异有关。
     

适应多领域多来源文本的汉语依存句法数据标注规范

近十年来,依存句法分析由于具有表示形式简单、灵活、分析效率高等特点,得到了学术界广泛关注。为了支持汉语依存句法分析研究,国内同行分别标注了几个汉语依存句法树库。然而,目前还没有一个公开、完整、系统的汉语依存句法数据标注规范,并且已有的树库标注工作对网络文本中的特殊语言现象考虑较少。为此,该文充分参考了已有的数据标注工作,同时结合实际标注中遇到的问题,制定了一个新的适应多领域多来源文本的汉语依存句法数据标注规范。我们制定规范的目标是准确刻画各种语言现象的句法结构,同时保证标注一致性。利用此规范,我们已经标注了约3万句汉语依存句法树库。     

基于依存语法构建多视图汉语树库

树库是自然语言处理中一项重要的基础资源,现有树库基本上都是单视图树,支持短语结构语法或者依存语法。该文提出一套基于依存语法的多视图汉语树库标注体系,仅需标注中心语和语法角色两类信息,之后可以自动地推导出描述句法结构所需的短语结构功能和层次信息,从而可以在不增加标注工作量的前提下获得更多语法信息。基于该体系,构建了北京大学多视图汉语树库(PMT)1.0版,含有64000句、140万词,支持短语结构语法和依存语法两个视图。     

Supervised collaboration for syntactic annotation of Quranic Arabic

The Quranic Arabic Corpus (http://corpus.quran.com) is a collaboratively constructed linguistic resource initiated at the University of Leeds, with multiple layers of annotation including part-of-speech tagging, morphological segmentation (Dukes and Habash 2010) and syntactic analysis using dependency grammar (Dukes and Buckwalter 2010). The motivation behind this work is to produce a resource that enables further analysis of the Quran, the 1,400 year-old central religious text of Islam. This project contrasts with other Arabic treebanks by providing a deep linguistic model based on the historical traditional grammar known as i′rāb (?????). By adapting this well-known canon of Quranic grammar into a familiar tagset, it is possible to encourage online annotation by Arabic linguists and Quranic experts. This article presents a new approach to linguistic annotation of an Arabic corpus: online supervised collaboration using a multi-stage approach. The different stages include automatic rule-based tagging, initial manual verification, and online supervised collaborative proofreading. A popular website attracting thousands of visitors per day, the Quranic Arabic Corpus has approximately 100 unpaid volunteer annotators each suggesting corrections to existing linguistic tagging. To ensure a high-quality resource, a small number of expert annotators are promoted to a supervisory role, allowing them to review or veto suggestions made by other collaborators. The Quran also benefits from a large body of existing historical grammatical analysis, which may be leveraged during this review. In this paper we evaluate and report on the effectiveness of the chosen annotation methodology. We also discuss the unique challenges of annotating Quranic Arabic online and describe the custom linguistic software used to aid collaborative annotation.     

Ensuring annotation consistency and accuracy for Vietnamese treebank

Treebanks are important resources for researchers in natural language processing. They provide training and testing materials so that different algorithms can be compared. However, it is not a trivial task to construct high-quality treebanks. We have not yet had a proper treebank for such a low-resource language as Vietnamese, which has probably lowered the performance of Vietnamese language processing. We have been building a consistent and accurate Vietnamese treebank to alleviate such situations. Our treebank is annotated with three layers: word segmentation, part-of-speech tagging, and bracketing. We developed detailed annotation guidelines for each layer by presenting Vietnamese linguistic issues as well as methods of addressing them. Here, we also describe approaches to controlling annotation quality while ensuring a reasonable annotation speed. We specifically designed an appropriate annotation process and an effective process to train annotators. In addition, we implemented several support tools to improve annotation speed and to control the consistency of the treebank. The results from experiments revealed that both inter-annotator agreement and accuracy were higher than 90%, which indicated that the treebank is reliable.     

汉语句法树库标注体系

语料库的句法标注是语料库语言学研究的前沿课题。本文在研究和总结国内外句法树库标注实践的基础上,提出了一套汉语真实文本的句法树标注体系。它以完整的层次结构树为基础,对句法树上的每个非终结符节点都给出两个标记:成分标记和关系标记,形成双标记集的句法信息描述体系。目前,这两个标记集分别包含了16和27个标记,对汉语句子的不同句法组合的外部功能分布和内部组合特点进行了详细描述。在此基础上,我们开发完成了100万词规模的汉语句法树库TCT,对其中各种复杂语言现象的标注实践显示了这套标注体系具有很好的信息覆盖率和语料适应性。     

A segment-based annotation tool for Korean treebanks with minimal human intervention

In this paper, we propose a segment-based annotation tool providing appropriate interactivity between a human annotator and an automatic parser. The proposed annotation tool provides the preview of a complete sentence structure suggested by the parser, and updates the preview whenever the annotator cancels or selects each segmentation point. Thus, the annotator can select the proper sentence segments maximizing parsing accuracy and minimizing human intervention. Experimental results show that the proposed tool allows the annotator to be able to reduce human intervention by approximately 39% compared with manual annotation. Sejong Korean treebank, one of the large scale treebanks, was constructed with the proposed annotation tool.     

短语结构树库向依存结构树库转化研究

汉语依存树库的建设相对其他语言如英语,在规模和质量上还有一些差距。树库标注需要付出很大的人力物力,并且保证树库质量也比较困难。该文尝试通过规则和统计相结合的方法,将宾州汉语短语树库Penn Chinese Treebank转化为哈工大依存树库HIT-IR-CDT的体系结构,从而增大现有依存树库的规模。将转化后的树库加入HIT-IR-CDT,训练和测试依存句法分析器的性能。实验表明,加入少量经转化后的树库后,依存句法分析器的性能有所提高;但加入大量树库后,性能反而下降。经过细致分析,作为一种利用多种树库提高依存句法分析器性能的方法,短语转依存还存在很多需要深入研究的方面。     

Annotation of semantic roles for the Turkish Proposition Bank

In this work, we report large-scale semantic role annotation of arguments in the Turkish dependency treebank, and present the first comprehensive Turkish semantic role labeling (SRL) resource: Turkish Proposition Bank (PropBank). We present our annotation workflow that harnesses crowd intelligence, and discuss the procedures for ensuring annotation consistency and quality control. Our discussion focuses on syntactic variations in realization of predicate-argument structures, and the large lexicon problem caused by complex derivational morphology. We describe our approach that exploits framesets of root verbs to abstract away from syntax and increase self-consistency of the Turkish PropBank. The issues that arise in the annotation of verbs derived via valency changing morphemes, verbal nominals, and nominal verbs are explored, and evaluation results for inter-annotator agreement are provided. Furthermore, semantic layer described here is aligned with universal dependency (UD) compliant treebank and released to enable more researchers to work on the problem. Finally, we use PropBank to establish a baseline score of 79.10 F1 for Turkish SRL using the mate-tool (an open-source SRL tool based on supervised machine learning) enhanced with basic morphological features. Turkish PropBank and the extended SRL system are made publicly available.     

基于词对依存分类的藏语树库半自动构建研究华

依据依存句法理论,该文制订了藏语句法标注体系及层次结构。通过分析构建藏语依存树库中存在的问题,提出了半自动依存树库构建模式,针对藏语特性提出了融合丰富特征的词对依存分类模型和依存边标注模型,实现了依存树库构建可视化工具,校对构建了1.1万句藏语依存句法树后,在基线系统下经实验验证,依存识别正确率提高了3%,使构建藏语依存树库工作取得了有效进展。     

Large aligned treebanks for syntax-based machine translation

We present a collection of parallel treebanks that have been automatically aligned on both the terminal and the non-terminal constituent level for use in syntax-based machine translation. We describe how they were constructed and applied to a syntax- and example-based machine translation system called Parse and Corpus-Based Machine Translation (PaCo-MT). For the language pair Dutch to English, we present non-terminal alignment evaluation scores for a variety of tree alignment approaches. Finally, based on the parallel treebanks created by these approaches, we evaluate the MT system itself and compare the scores with those of Moses, a current state-of-the-art statistical MT system, when trained on the same data.     

A feature-based approach to better automatic treebank conversion

In the field of constituency parsing, there exist multiple human-labeled treebanks which are built on non-overlapping text samples and follow different annotation standards. Due to the extreme cost of annotating parse trees by human, it is desirable to automatically convert one treebank (called source treebank) to the standard of another treebank (called target treebank) which we are interested in. Conversion results can be manually corrected to obtain higher-quality annotations or can be directly used as additional training data for building syntactic parsers. To perform automatic treebank conversion, we divide constituency parses into two separate levels: the part-of-speech (POS) and syntactic structure (bracketing structures and constituent labels), and conduct conversion on these two levels respectively with a feature-based approach. The basic idea of the approach is to encode original annotations in a source treebank as guide features during the conversion process. Experiments on two Chinese treebanks show that our approach can convert POS tags and syntactic structures with the accuracy of 96.6 and 84.8 %, respectively, which are the best reported results on this task.     

Annotation of multiword expressions in the Prague dependency treebank

We describe annotation of multiword expressions (MWEs) in the Prague dependency treebank, using several automatic pre-annotation steps. We use subtrees of the tectogrammatical tree structures of the Prague dependency treebank to store representations of the MWEs in the dictionary and pre-annotate following occurrences automatically. We also show a way to measure reliability of this type of annotation.     

汉语中介语的依存句法标注规范及标注实践

汉语中介语是伴随着汉语国际教育产生的,随着汉语学习在全球的不断开展,汉语中介语的规模不断增长,由于这些语料在语言使用上有其独特性,使得中介语成为语言信息处理和智能语言辅助学习的独特资源。依存语法分析是语言信息处理的重要步骤,英语中介语的依存语法标注语料已经有很好的应用,目前汉语中介语语料库对句法的关注度较低,缺乏一个充分考虑汉语中介语特点的依存句法标注规范。该文着眼于汉语中介语的依存句法标注语料库的建构,探讨依存标注规范,在充分借鉴国际通用依存标注体系(Universal Dependencies)的基础上,制定了汉语中介语的依存标注规范,并进行了标注实践,形成了一个包括汉语教学语法点的中介语依存语料库。     

维吾尔语依存树库构建及统计分析

本着构建维吾尔语依存树库的目的,该文根据黏着性语言的结构特点及其在依存属性中对依存角色的影响,提出构建维吾尔语依存树库时需要考虑的几点要素。其包含依存粒度的确定、维吾尔语依存关系、标注原则、依存树结构以及标注工具的设计与实现。然后根据《维吾尔语依存树库标注手册》人工标注了3 400多条句子并从三个角度对依存树库信息做了统计分析。     

支持背景知识的多维端到端短语识别算法研究

目前,实体识别与依存关系分析,采用的主要是基于监督学习的深度端到端方法.这种方法存在两个问题:不能引入背景知识;不能识别出自然语言的多粒度、嵌套特征.为了解决以上问题,提出了基于短语窗口的依存句法标注规则,并标注了中文短语窗口数据集(CPWD),同时设计了配套的多维端到端短语识别模型(MDM模型).该标注规则以短语为最...     

A dependency annotation scheme for Bangla treebank

Dependency grammar is considered appropriate for many Indian languages. In this paper, we present a study of the dependency relations in Bangla language. We have categorized these relations in three different levels, namely intrachunk relations, interchunk relations and interclause relations. Each of these levels is further categorized and an annotation scheme has been developed. Both syntactic and semantic features have been taken into consideration for describing the relations. In our scheme, there are 63 such syntactico–semantic relations. We have verified the scheme by tagging a corpus of 4167 Bangla sentences to create a treebank (KGPBenTreebank).