结合结构下文及词汇信息的汉语句法分析方法

针对句法分析中上下文无关语法模型对句子信息利用的不足,通过融入结构下文和部分词汇信息,提出两种基于概率上下文无关语法模型的短语结构消歧方法,以达到消解结构歧义的目的;引入分层分析的算法,通过损失一定的时间效率使得在提高分析准确率的同时保证分析结果的全面性。实验结果表明,融入结构下文及词汇信息的汉语句法分析方法,利用了更多的句子信息,与上下文无关语法相比有着更强的消歧能力。     

基于二元组合文法的歧义消解模型

本文论述了一种基于二元组合文法的汉语句法结构分析的消除歧义方法。首先给出了二元组合文法的基本概念以及基本思想,然后研究了概率上下文无关文法独立性假设的限性,并针对局限性引入了基于二元组合文法的上下文相关的概率信息,同时提出了一种新的计算分值模式。实验结果证明,这种方法对句法分析过程中的歧义消解是有效的。     

中文分词中组合型切分歧义的消解研究

针对中文自动分词中组合型歧义消解难的问题,提出了一种新的切分算法来对组合型歧义字段进行消歧。该算法首先自动从训练语料中提取歧义字段的上下文信息来建立规则库,然后利用C-SVM模型结合规则对组合型歧义字段进行歧义消解。最后以1998年1月《人民日报》语料中出现的组合型歧义字段进行训练和测试,实验显示消歧的平均准确率达89.33%。     

基于规则的模糊语句分析

针对当前常用的处理模糊语句的方法进行了分析,介绍了概率和规则相结合的消除模糊的方法:概率上下文无关文法,比较了几种方法的优缺点。     

基于评价的英语句法结构消歧和自我评价的规则校正

句法结构歧义是句法分析过程中最主要也是最难解决的问题之一．使用了一种组合型的概率评价函数解决英语句法分析过程中所面临的结构歧义问题，综合考虑了句法分析时的结构特性和上下文信息．并使用基于规则的句法分析后校正机制对错误的分析结果进行进一步的校正．组合型的概率评价函数为更精细的句法分析提供了一个可以扩充的统计模型，这种方法也可以适用于其它语言的句法分析．实验结果表明，这种方法具有较高的精确率和召回率。     

层级分类概率句法分析

对已有的句法分析中引入知识的方法进行了归纳分析,认为多种句法分析方法都可被看作是基于特征标记的分类,然后分析了其中的欠分类和过分类问题.在此基础上,提出一种层级分类短语结构文法和一种层级分类概率句法分析方法(hierarchically classified probabilistic context-free grammar),并设计了一种通过对实例进行聚类来消除句法规则的分类歧义方法.还进一步将层级分类扩展到概率上下文相关句法分析方法,利用上下文相关性的层级分类来解决引入上下文相关时的数据稀疏性问题.通过上述一系列方法有效地克服了过分类与前分类之间的矛盾.     

融合上下文信息的多尺度贝叶斯图像分割

提出了一种融合上下文信息的多尺度贝叶斯图像分割算法，基于多尺度MRF图像模型，将模型中各结点的邻域结点类别信息抽象为上下文，求得结点的后验边缘概率之后，在各尺度融合表征了同一尺度内及相邻尺度的邻域信息的上下文，结点在相邻结点信息的指导下，得到的分割结果在均匀区域内部及区域边界都大为改善，而且没有增加模型的复杂度，算法仍然是快速的、非迭代的．融合过程中的参数采用EM算法估计．分析和实验结果表明算法是有效的．     

基于上下文语境的词义消歧方法

针对传统词义消歧方法面临的数据稀疏问题,提出一种基于上下文语境的词义消歧方法。该方法假设同一篇文章中的句子之间共享一些相同的话题,首先,抽取在同一篇文章中包含相同歧义词的句子,这些句子可以作为歧义句的上下文语境,为其中的一个歧义句子提供消歧知识;其次,通过一种无监督的词义消歧方法进行词义消歧。在真实的语料上实验结果表明,使用2个上下文语境句子,窗口大小为1时,该方法的消歧准确率比基线方法(OrigDisam)提高了3.26%。     

语义信息集成中上下文仲裁器的设计与实现

已有的基于模式映射的语义信息集成能够解决分布数据源之间的模式异构,对于普遍存在的上下文异构则无法解决.首先提出一种将暗含的上下文语义进行形式化描述的方法,然后在基于模式映射的语义信息集成基础上,增加上下文仲裁器以自动检测和解决上下文异构.详细介绍了上下文仲裁器的工作原理、设计思想与实现细节.     

新的普适计算环境下上下文缓存置换算法

由于普适计算环境高度动态的特点以及无线环境连接易中断、传输速度低等方面的约束,使得普适计算应用对于上下文的访问开销非常大。为此,首先给出一个上下文感知系统框架,然后提出了一种基于规则的上下文缓存置换算法--RCRA,算法根据上下文的被访问概率、上下文时效性及历史访问次数决定是否将其置换出缓存。当有新的上下文需要进入缓存时执行该算法,以保证缓存中的上下文最新且最有价值。实验表明,RCRA不仅在命中率方面有较大提高,而且能够有效降低上下文访问的开销。算法应用于基于推理规则的上下文感知系统中,具有良好的可实用性。     

词汇化句法分析与子语类框架获取的互动方法

概率句法分析器(PCFG Parser)是基于概率规则集的上下文无关文法的句法分析器。规则集主要是针对词类和短语类。然而事实上,词性相同而词汇不同,其所常用的句法规则也通常不同。目前NLP研究的一个趋势和热点就是词汇化的句法分析。针对概率句法分析独立性假设中缺乏词汇化的缺陷,本文将谓语动词的子语类信息与概率句法分析结合起来,提出一种基于动词子语类信息的词汇化概率句法分析方法。论文建立了基于汉语动词子语类框架的统计句法分析模型,并且针对动词子语类框架难以获取的问题,提出一种词汇化概率句法分析与动词子语类框架获取的互动方法。实验利用这种互动的方法获取了汉语中十个常用高频动词的概率化子语类信息,并结合原有的概率句法分析器PCFG实现了一个基于动词子语类信息的概率句法分析器原型系统S-PCFG。实验证明了基于动词子语类信息的概率句法分析对自然语言句法分析的准确率和速度均有所提高。同时分析了新的概率句法分析器的不足之处,为进一步的改进提供条件。     

Latent Variable Model for Estimation of Distribution Algorithm Based on a Probabilistic Context-Free Grammar

Estimation of distribution algorithms are evolutionary algorithms using probabilistic techniques instead of traditional genetic operators. Recently, the application of probabilistic techniques to program and function evolution has received increasing attention, and this approach promises to provide a strong alternative to the traditional genetic programming techniques. Although a probabilistic context-free grammar (PCFG) is a widely used model for probabilistic program evolution, a conventional PCFG is not suitable for estimating interactions among nodes because of the context freedom assumption. In this paper, we have proposed a new evolutionary algorithm named programming with annotated grammar estimation based on a PCFG with latent annotations, which allows this context freedom assumption to be weakened. By applying the proposed algorithm to several computational problems, it is demonstrated that our approach is markedly more effective at estimating building blocks than prior approaches.      

汉语口语对话系统中语义分析的消歧策略

框架语义分析是目前汉语口语对话系统中常用的语义解析方法,本文分析了语义分析过程中容易产生的两种典型歧义现象- 结构歧义和语义关系歧义。并针对这两种歧义结构,分别提出基于语义PCFG模型的结构歧义消歧策略以及基于语义期待模型EM的语义关系歧义消歧策略,并给出了有效的消歧算法。实验结果表明综合运用本文提出的消歧策略后,基线系统理解模块的句子语义分析正确率大大提高,从原来的7517 %上升到9115 % ,而且标志语义单元理解率的三项指标,准确率,召回率和精度也平均提高了10 %。     

Probabilistic Alias Analysis of Executable Code

In this paper we present a method for flow-sensitive, context-insensitive probabilistic alias analysis at the assembly level. A memory disambiguation algorithm is also developed for revealing the probability of two registers holding the same memory location. The alias analysis and memory disambiguation algorithms are implemented based on the Diablo post-link optimizer. Experimental results show that the technique can estimate the probabilities that registers refer to the same memory address in benchmark programs with an overall average error of about 6.8%. The post-link optimizer can leverage the obtained quantitative information to facilitate aggressive analyses and optimizations.     

A Note on the Expressive Power of Probabilistic Context Free Grammars

We examine the expressive power of probabilistic context free grammars (PCFGs), with a special focus on the use of probabilities as a mechanism for reducing ambiguity by filtering out unwanted parses. Probabilities in PCFGs induce an ordering relation among the set of trees that yield a given input sentence. PCFG parsers return the trees bearing the maximum probability for a given sentence, discarding all other possible trees. This mechanism is naturally viewed as a way of defining a new class of tree languages. We formalize the tree language thus defined, study its expressive power, and show that the latter is beyond context freeness. While the increased expressive power offered by PCFGs helps to reduce ambiguity, we show that, in general, it cannot be decided whether a PCFG removes all ambiguities.     

Generalized queries on probabilistic context-free grammars

Probabilistic context-free grammars (PCFGs) provide a simple way to represent a particular class of distributions over sentences in a context-free language. Efficient parsing algorithms for answering particular queries about a PCFG (i.e., calculating the probability of a given sentence, or finding the most likely parse) have been developed and applied to a variety of pattern-recognition problems. We extend the class of queries that can be answered in several ways: (1) allowing missing tokens in a sentence or sentence fragment, (2) supporting queries about intermediate structure, such as the presence of particular nonterminals, and (3) flexible conditioning on a variety of types of evidence. Our method works by constructing a Bayesian network to represent the distribution of parse trees induced by a given PCFG. The network structure mirrors that of the chart in a standard parser, and is generated using a similar dynamic programming approach. We present an algorithm for constructing Bayesian networks from PCFGs, and show how queries or patterns of queries on the network correspond to interesting queries on PCFGs. The network formalism also supports extensions to encode various context sensitivities within the probabilistic dependency structure     

Variable-Length Unit Selection in TTS Using Structural Syntactic Cost

This paper presents a variable-length unit selection scheme based on syntactic cost to select text-to-speech (TTS) synthesis units. The syntactic structure of a sentence is derived from a probabilistic context-free grammar (PCFG), and represented as a syntactic vector. The syntactic difference between target and candidate units (words or phrases) is estimated by the cosine measure with the inside probability of PCFG acting as a weight. Latent semantic analysis (LSA) is applied to reduce the dimensionality of the syntactic vectors. The dynamic programming algorithm is adopted to obtain a concatenated unit sequence with minimum cost. A syntactic property-rich speech database is designed and collected as the unit inventory. Several experiments with statistical testing are conducted to assess the quality of the synthetic speech as perceived by human subjects. The proposed method outperforms the synthesizer without considering syntactic property. The structural syntax estimates the substitution cost better than the acoustic features alone     

面向数据的句法分析消歧

面向数据的分析技术(Data-Oriented Parsing,DOP)是一种概率分析策略,其概率模型的主要目的在于为一个给定的句子找到最可能的分析,即分析消歧。实际上,有关算法计算复杂度的大量研究证明,该类消歧问题属于NP-完全问题。因此,为有效实现最可能的分析,国外学者提出许多近似分析算法。本文主要论述在 DOP 框架中,基于 Monte Carlo 方法找到最可能分析的近似分析算法,并说明该方法可在合理的算法时间代价范围内实现,而且在统计上受控,以确保所获得的近似解确实对应着分析消歧后的精确解。