一种处理结构化输入输出的中文句法分析方法

中文句法结构复杂,特征维数较高,目前已知最好的汉语句法分析效果与其他西方语言相比还有一定的差距。为进一步提高中文句法分析的效率和精度,该文提出一种采用二阶范数软间隔优化的结构化支持向量机(StructuralSupportVectorMachines,StructuralSVMs)方法对基于短语结构的中文句法进行分析,通过构造结构化特征函数ψ(x,y),体现句法树的输入信息,并根据中文句子本身具有的强相关性,在所构造的ψ(x,y)中增加中文句法分析树中父节点的信息,使ψ(x,y)包含了更加丰富的结构信息。在宾州中文树库PCTB上的实验结果表明,该文方法与经典结构化支持向量机方法以及BerkeleyParser相比可取得较好的效果。     

基于主动学习的中文依存句法分析

目前依存句法分析仍主要采用有指导的机器学习方法,即需要大规模高质量的树库作为训练语料,而现阶段中文依存树库资源相对较少,树库标注又是一件费时费力的工作。面对大量未标注语料,该文将主动学习应用到中文依存句法分析,优先选择句法模型预测不准的实例交由人工标注。该文提出并比较了多种衡量依存句法模型预测可信度的准则。实验表明,一方面,与随机选择标注实例相比,当使用相同数目训练实例时,主动学习使中文依存分析性能最高提升0.8%;另一方面,主动学习使依存分析达到相同准确率时只需标注更少量实例,人工标注量最多可减少30%。     

越南语短语树到依存树的转换研究

依存句法分析是自然语言处理的一个关键环节,目前对于越南语短语结构树的研究比较多,而依存结构树的研究就显得十分薄弱。提出了一种新的方法,尝试结合越南语的语言特点和语法特征,利用中心子节点过滤表的思想与统计的方法将越南语的短语结构树转换成依存结构树。首先依据中文依存关系标注体系与越南语的语法规则,制定出依存关系列表;然后结合越南语的语言特点,制定出中心子节点过滤表,利用中心子节点过滤表的思想进行初步转化;最后使用依存关系标注器来进行依存关系标注。基于转换后得到的依存结构树,利用MSTParser工具进一步训练得到更多的越南语依存结构树。对实验结果进行了抽样评估,树库转换的准确率达到了89.4%,较好地解决了越南语由短语树到依存树的转换问题。     

基于树库转换的藏语依存句法树库构建方法

构建藏语依存树库是实现藏语句法分析的重要基础,对藏语本体研究和信息处理具有重要价值。基于此,该文提出了一种基于树库转换的藏语依存树库构建方法。该方法首先扩充了前期构建的藏语短语结构树库,然后根据藏语短语结构树和依存树的特征设计树库转换规则,实现藏语短语结构树到依存结构树的初步转换,最后对自动转换结果进行人工校验,得到了2.2万句藏语依存树。为了对转换结果做出量化评价,该文抽取了依存树库中5%的依存树,对其依存关系进行校验和统计,最终依存关系的准确率达到89.36%,中心词的准确率达到92.09%。此外,该文使用基于神经网络的句法分析模型验证了依存树库的有效性。在该模型上,UAS值和LAS值分别达到83.62%和81.90%。研究证明,使用半自动的树库转换方法能够有效地完成藏语依存树库构建工作。     

短语结构树库向依存结构树库转化研究

汉语依存树库的建设相对其他语言如英语,在规模和质量上还有一些差距。树库标注需要付出很大的人力物力,并且保证树库质量也比较困难。该文尝试通过规则和统计相结合的方法,将宾州汉语短语树库Penn Chinese Treebank转化为哈工大依存树库HIT-IR-CDT的体系结构,从而增大现有依存树库的规模。将转化后的树库加入HIT-IR-CDT,训练和测试依存句法分析器的性能。实验表明,加入少量经转化后的树库后,依存句法分析器的性能有所提高;但加入大量树库后,性能反而下降。经过细致分析,作为一种利用多种树库提高依存句法分析器性能的方法,短语转依存还存在很多需要深入研究的方面。     

句式结构树库的自动构建研究

句式结构树库是以句本位语法为理论基础构建的句法资源，对汉语教学以及句式结构自动句法分析等研究具有重要意义。目前已有的句式结构树库语料主要来源于教材领域，其他领域的标注数据较为缺乏，如何高效地扩充高质量的句法树库是值得研究的问题。人工标注句法树库费时费力，树库质量也难以保证，为此，该文尝试通过规则的方法，将宾州中文树库(CTB)转换为句式结构树库，从而扩大现有句式结构树库的规模。实验结果表明，该文提出的基于树库转换规则的方法是有效的。     

基于序列标注的中文依存句法分析方法

提出了一种基于序列标注模型的中文依存句法分析方法.该方法将依存句法分析转化成序列标注问题,利用条件随机场CRF(Conditional Random Field)建立序列标注模型.在宾州中文树库的测试中,达得了76.59%的依存关系准确率,句子准确率也达到了23.5%.同时我们改进了Viterbi算法,使得依存关系的准确率提高了近2个百分点,句子准确率提高了近3.5个百分点.     

基于字符的中文分词、词性标注和依存句法分析联合模型

目前,基于转移的中文分词、词性标注和依存句法分析联合模型存在两大问题: 一是任务的融合方式有待改进;二是模型性能受限于全标注语料的规模。针对第一个问题,该文利用词语内部结构将基于词语的依存句法树扩展成了基于字符的依存句法树,采用转移策略,实现了基于字符的中文分词、词性标注和依存句法分析联合模型;依据序列标注的中文分词方法,将基于转移的中文分词处理方案重新设计为4种转移动作: Shift_S、Shift_B、Shift_M和Shift_E,同时能够将以往中文分词的研究成果融入联合模型。针对第二个问题,该文使用具有部分标注信息的语料,从中抽取字符串层面的n-gram特征和结构层面的依存子树特征融入联合模型,实现了半监督的中文分词、词性标注和依存句法分析联合模型。在宾州中文树库上的实验结果表明,该文的模型在中文分词、词性标注和依存分析任务上的F1值分别达到了98.31%、94.84%和81.71%,较单任务模型的结果分别提升了0.92%、1.77%和3.95%。其中,中文分词和词性标注在目前公布的研究结果中取得了最好成绩。     

基于句法的统计机器翻译模型与方法

该文总结了我们近几年来在基于句法的统计机器翻译方面所做的研究工作,特别是基于源语言句法的一系列统计机器翻译模型与方法,具体包括 基于最大熵括号转录语法的翻译模型,基于源语言短语结构树的树到串翻译模型及其相应的基于树的翻译方法,基于森林的翻译方法和句法分析与解码一体化翻译方法,基于源语言依存树的翻译模型。     

基于子树的确定性依存分析方法

确定性依存句法分析(又称为基于动作的依存句法分析或基于转换的依存句法分析),由于采用了贪婪的搜索策略,它是一类比较有效率的依存句法分析方法.在一种主流的确定性依存句法分析方法-Yamada-Matsumoto算法的基础上,通过改进其转换模型,提出了一种基于子树的确定性依存句法分析方法.在新的方法中,引入了子树的结构,因而能够更加灵活地确定词与词间的依存关系.最后,通过在中文宾州树库上的实验,证明了这个新方法的有效性.     

一个基于GLR算法的英汉机器翻译浅层句法分析器

浅层句法分析是指短语级的自然语言句法分析。在研制MatLink英汉机器翻译系统的过程中,提出了扩充的CFG文法用于描述英语短语句法,并改进了GLR算法,设计实现了用于英汉翻译的英语浅层句法分析器。该分析器采用多出口的分析表结构,引入符号映射函数实现短语边界的自动识别,用孩子兄弟树描述短语的句法结构,并通过短语转换模式实现源语言向目标语言的短语级转换。最后,通过对一个实例句子的分析阐述了该浅层句法分析器的设计思想和工作过程。     

Using Decision Trees to Construct a Practical Parser

This paper describes a novel and practical Japanese parser that uses decision trees. First, we construct a single decision tree to estimate modification probabilities; how one phrase tends to modify another. Next, we introduce a boosting algorithm in which several decision trees are constructed and then combined for probability estimation. The constructed parsers are evaluated using the EDR Japanese annotated corpus. The single-tree method significantly outperforms the conventional Japanese stochastic methods. Moreover, the boosted version of the parser is shown to have great advantages; (1) a better parsing accuracy than its single-tree counterpart for any amount of training data and (2) no over-fitting to data for various iterations. The presented parser, the first non-English stochastic parser with practical performance, should tighten the coupling between natural language processing and machine learning.     

汉语短语的自动划分和标注

考虑到传统的基于规则的汉语分析器对大规模真实文本的分析所遇到的困难, 本文在使用统计方法进行汉语自动句法分析方面作了一些探索, 提出了一套基于统计的汉语短语自动划分和标注算法, 它分为预测划分点、括号匹配和分析树生成等三个处理阶段, 其间利用了从人工标注的树库中统计得到的各种数据进行自动句法排歧, 最终得到一棵最佳句法分析树, 从而可以自顶向下地完成对一句句子的短语自动划分和标注, 对一千多句句子的封闭测试结果表明, 短语划分的正确率约为86%, 短语标注的正确率约为92%, 处理效果还是比较令人满意的。     

高性能的XML解析器OnceXMLParser

XML(extensible markup language)解析器是分析、处理XML文档的基础软件.研究高性能验证型XML解析器的实现.开发了支持3种解析模型的XML解析器OnceXMLParser,该解析器通过了严格的XML兼容性测试和API兼容性测试.OnceXMLParser具有轻量级体系结构并进行了多方面的性能优化,包括高效的词法分析、基于统计分析的自动机实现、合理的资源分配策略以及语言层次上的优化.性能测试结果表明,OnceXMLParser具有出色的解析性能.     

Improving Shift‐Reduce Phrase‐Structure Parsing with Constituent Boundary Information

Shift‐reduce parsing enjoys the property of efficiency because of the use of efficient parsing algorithms like greedy/deterministic search and beam search. In addition, shift‐reduce parsing is much simpler and easy to implement compared with other parsing algorithms. In this article, we explore constituent boundary information to improve the performance of shift‐reduce phrase‐structure parsing. In previous work, constituent boundary information has been used to speed up chart parsers successfully. However, whether it is useful for improving parsing accuracy has not been investigated. We propose two different models to capture constituent boundary information, based on which two sets of novel features are designed for a shift‐reduce parser. The first model is a boundary prediction model that uses a classifier to predict the boundaries of constituents. We use automatically parsed data to train the classifier. The second one is a Tree Likelihood Model that measures the validity of a constituent by its likelihood which is calculated on automatically parsed data. Experimental results show that our proposed method outperforms a strong baseline by 0.8% and 1.6% in F‐score on English and Chinese data, respectively, achieving the competitive parsing accuracies on Chinese (84.8%) and English (90.8%). To our knowledge, this is the first time for shift‐reduce phrase‐structure parsing to advance the state‐of‐the‐art with constituent boundary information.     

Efficient error-correcting parsing for (attributed and stochastic) tree grammars

By representing a tree in terms of two related strings, a minimum-distance error-correcting parser (MDECP) for an expansive tree grammar and a maximum-likelihood error-correcting parser (MLECP) for a stochastic and attributed expansive tree grammar are proposed in this paper. The MDECP allows the use of an optimization technique to speed up the parsing. The MLECP of the attributed tree grammar takes both the structure errors and the semantic errors into consideration, and has a high parsing efficiency. As an example, the problem of handprinted-character recognition is given. It shows that a small training set could result in a powerful and efficient classifier.     

基于自然语言的分拣机器人解析器技术研究

工业机器人通常采用特定的机器人语言进行示教编程与控制,对于操作人员需要具有较高专业与技能要求,并且示教周期长导致工作效率降低。为了提高工业机器人使用效率与易用性,提出一种基于受限自然语言解析器的设计方法。该系统通过对受限自然语言进行词法解析、语法解析、语义解析,得到所需求的工作意图,然后与实时生成的三维空间语义地图进行匹配,结合机械臂轨迹规划,生成能够完成工作任务的机器人作业程序,并完成了机器人作业程序的解析与实际机械臂的控制。通过实验证明设计的基于受限自然语言处理的分拣机器人解析器能够正确解析自然语言命令,实现对机械臂的控制。     

Syntax optimization for and parsing of patterns

Theory and algorithm for optimization of a directed and labeled tree are presented. Their application for optimizing any finite pattern grammar represented in the form of a tree is discussed. Tree optimization leads to loss information which is essential for identification of patterns. Special technique for preserving this information has been suggested.Finally, outlines of two different algorithms for the parsing of patterns are included. The tree parser uses the optimized tree and the table-driven parser uses the optimized syntax stored in four separate tables.     

How to design a connectionist holistic parser

Connectionist holistic parsing offers a viable and attractive alternative to traditional algorithmic parsers. With exposure to a limited subset of grammatical sentences and their corresponding parse trees only, a holistic parser is capable of learning inductively the grammatical regularity underlying the training examples that affects the parsing process. In the past, various connectionist parsers have been proposed. Each approach had its own unique characteristics, and yet some techniques were shared in common. In this article, various dimensions underlying the design of a holistic parser are explored, including the methods to encode sentences and parse trees, whether a sentence and its corresponding parse tree share the same representation, the use of confluent inference, and the inclusion of phrases in the training set. Different combinations of these design factors give rise to different holistic parsers. In succeeding discussions, we scrutinize these design techniques and compare the performances of a few parsers on language parsing, including the confluent preorder parser, the backpropagation parsing network, the XERIC parser of Berg (1992), the modular connectionist parser of Sharkey and Sharkey (1992), Reilly's (1992) model, and their derivatives. Experiments are performed to evaluate their generalization capability and robustness. The results reveal a number of issues essential for building an effective holistic parser.     

DiSeg 1.0: The first system for Spanish discourse segmentation

Nowadays discourse parsing is a very prominent research topic. However, there is not a discourse parser for Spanish texts. The first stage in order to develop this tool is discourse segmentation. In this work, we present DiSeg, the first discourse segmenter for Spanish, which uses the framework of Rhetorical Structure Theory and is based on lexical and syntactic rules. We describe the system and we evaluate its performance against a gold standard corpus, divided in a medical and a terminological subcorpus. We obtain promising results, which means that discourse segmentation is possible using shallow parsing.