评价短语的倾向性分析研究

评价短语是评价因子之一,是汉语倾向性研究的重要组成部分。评价短语可以分为“评价词+评价词”、“修饰词+评价词”、“普通词+评价词”、“修饰词+普通词”、“普通词+普通词”5类。评价短语类型不同,采用的倾向性分析策略也不同。短语计算规则和短语评价词典的互动是该文采用的基本方法。在制定短语计算规则时应遵守共性与个性相结合的原则;建立短语评价词典时应遵循最小评价因子原则。实验证明,短语计算规则与短语词典的建立提高了倾向性分析系统的准确率,是一种行之有效的方法。     

汉英短语翻译对的自动抽取

介绍从平行语料库中如何抽取双语短语翻译对。首先用统计模型正则期望从汉语专利语料库中抽取汉语短语。抽取的短语利用统计知识和语言学知识来过滤,使得过滤后汉语短语的正确率较高;其次,利用词对齐工具Giza++从汉英平行语料库中抽取词汇对齐,在词汇对齐的基础上利用开源工具Moses抽取汉英短语对齐,根据短语对齐与抽取出的高质量汉语短语的交集来抽取候选的汉英互译的源语言短语;接着使用停用词、对数似然估计法LLR和上下文熵来对英语短语翻译进行过滤。实验结果表明,过滤后,抽取的汉语短语准确率为97.6%,汉英短语翻译对的准确率为92.4%。     

A novel retrieval approach reflecting variability of syntactic phrase representation

In this paper, we introduce variability of syntactic phrases and propose a new retrieval approach reflecting the variability of syntactic phrase representation. With variability measure of a phrase, we can estimate how likely a phrase in a given query would appear in relevant documents and control the impact of syntactic phrases in a retrieval model. Various experimental results over different types of queries and document collections show that our retrieval model based on variability of syntactic phrases is very effective in terms of retrieval performance, especially for long natural language queries.     

Phrase classes in two-level language models for ASR

In this work, we propose and compare two different approaches to a two-level language model. Both of them are based on phrase classes but they consider different ways of dealing with phrases into the classes. We provide a complete formulation consistent with the two approaches. The language models proposed were integrated into an Automatic Speech Recognition (ASR) system and evaluated in terms of Word Error Rate. Several series of experiments were carried out over a spontaneous human–machine dialogue corpus in Spanish, where users asked for information about long-distance trains by telephone. It can be extracted from the obtained results that the integration of phrases into classes when using the language models proposed leads to an improvement of the performance of an ASR system. Moreover, the obtained results seem to indicate that the history length with which the best performance is achieved is related to the features of the model itself. Thus, not all the models show the best results with the same value of history length.     

基于中心语块扩展的汉藏基本名词短语对的识别

该文提出汉藏基本名词短语对齐框架。从汉语基本名词短语出发,找藏文正确译文过程中,参考英汉短语对齐的方法,针对藏语的特殊性,提出基于中心语块扩展的藏语基本名词短语识别方法。提出词典与自动词对齐结果相结合的方法和基于序列相交的方法抽取藏语中心语块,再以扩展可信度为依据扩展中心语块。实验结果表明,基于序列相交的方法所抽取的汉藏基本名词短语对能够节省人工校正的工作量,有效辅助于汉藏基本名词短语库的建设。     

Formalisation method for the text expressed knowledge

This paper describes a method of knowledge representation as a set of text expressed statements. The method is based on the identification of word-categories/phrases and their semantic relationships within the observed statement. Furthermore, the identification of semantic relationships between words/phrases using wh-questions that clarify the role of the word/phrase in the relationship is described. A conceptual model of the computer system based on the formalization method of text-expressed knowledge is proposed. The subsystem text formalization is described in detail, especially its parts: syntactic analysis of the sentence, sentence formalization, phrase structure grammar and lexicon. The phrase structure grammar is formed by induction and it is used to generate the language of the formalized notation of a sentence. The derivation of grammar is based on the simple phrase structure grammar which was used for the syntactical analysis of informal language notation. In its base, the suggested method translates sentences of the informal language into formal language sentences which are generated by the derivated phrase structure grammar. Current limitations of the method that also set the path of its further development are shown. Next concrete steps in the development of the method are also described.     

基于自然标注信息和隐含主题模型的无监督文本特征抽取

术语和惯用短语可以体现文本特征。无监督的抽取特征词语对诸多自然语言处理工作起到支持作用。该文提出了“聚类-验证”过程,使用主题模型对文本中的字符进行聚类,并采用自然标注信息对提取出的字符串进行验证和过滤,从而实现了从未分词领域语料中无监督获得词语表的方法。通过优化和过滤,我们可以进一步获得了富含有术语信息和特征短语的高置信度特征词表。在对计算机科学等六类不同领域语料的实验中,该方法抽取的特征词表具有较好的文体区分度和领域区分度。     

基于统计和词典方法相结合的韩汉双语语料库名词短语对齐

韩汉双语语料库短语对齐对于基于实例的韩汉机器翻译系统具有重要意义,该文从韩国语名词短语结构特点出发,在基于统计和基于词典的词对齐方法进行试验分析的基础上,提出了基于词对齐位置信息的韩汉双语语料库名词短语对齐方法。该方法通过基于统计的方法获得词对齐位置信息,在此基础上利用基于词典方法的相似度计算进行词对齐校正;根据以上结果,该文通过韩国语名词短语左右边界规则抽取名词短语及其汉语译文,利用关联度度量方法进行过滤,实现名词短语对齐。实验结果表明,在较大规模语料库情况下,该方法取得了较好的短语对齐结果。     

有限语料汉蒙统计机器翻译调序方法研究

自统计机器翻译技术出现以来,调序一直是语序差异显著的语言对互译系统中的关键问题,基于大规模语料训练的调序方法得到了广泛研究。目前汉蒙双语语料资源十分有限,使得现有的依赖于大规模语料和语言学知识的调序方法难以取得良好效果。该文对已有的相关研究进行了分析,提出了在有限语料条件下的汉蒙统计机器翻译调序方法。该方法依据语言学知识获取对译文语序影响显著的短语类型,研究这些短语类型的调序方案,并融入已有的调序模型实现调序的优化。实验表明该方法在有限语料条件下的效果提升显著。     

自动提取含字母词语的领域新术语的研究

新术语的提取是中文信息处理领域的一个重要研究课题。针对现有提取方法的不足和很多专业术语表现为字母词语的特点,该文提出了一种综合统计技术和规则筛选的方法：基于长串优先和串频统计的思路进行文本切分,得到共现字符串,利用词语搭配规则进行过滤,经过领域词典及评价函数的筛选,提取出领域新术语。该方法可发现包含字母词语、专业术语等未登录词在内的频率大于等于2的任意长度的专指语义串、短语和词。实验表明了该方法的有效性及新术语的准确率分布特征。     

结合统计特征和图模型的半监督式中文关键短语抽取方法

关键短语抽取,即从文档中抽取能够表达文档主题和内容的关键短语集合,对于信息检索和文档分类等文本处理任务具有重要意义。然而,现有文献缺乏针对中文特点的关键短语抽取算法的研究。为此,该文提出了一种半监督式中文关键短语抽取模型,该模型采用预训练语言模型来表征短语及文章,以减少算法对大量标注训练数据的依赖;进而提出图模型描述候选短语间的相似性空间并迭代计算各短语的重要度;同时结合了多项统计特征来进一步提高短语评估的准确率。对比实验表明,该文提出的方法在中文关键短语抽取方面比基线方法具有明显的提升效果。     

Bilingual LSA-based adaptation for statistical machine translation

We propose a novel approach to cross-lingual language model and translation lexicon adaptation for statistical machine translation (SMT) based on bilingual latent semantic analysis. Bilingual LSA enables latent topic distributions to be efficiently transferred across languages by enforcing a one-to-one topic correspondence during training. Using the proposed bilingual LSA framework, model adaptation can be performed by, first, inferring the topic posterior distribution of the source text and then applying the inferred distribution to an n-gram language model of the target language and translation lexicon via marginal adaptation. The background phrase table is enhanced with the additional phrase scores computed using the adapted translation lexicon. The proposed framework also features rapid bootstrapping of LSA models for new languages based on a source LSA model of another language. Our approach is evaluated on the Chinese–English MT06 test set using the medium-scale SMT system and the GALE SMT system measured in BLEU and NIST scores. Improvement in both scores is observed on both systems when the adapted language model and the adapted translation lexicon are applied individually. When the adapted language model and the adapted translation lexicon are applied simultaneously, the gain is additive. At the 95% confidence interval of the unadapted baseline system, the gain in both scores is statistically significant using the medium-scale SMT system, while the gain in the NIST score is statistically significant using the GALE SMT system.     

依存树到串模型中引入双语短语的三种方法

依存树到串模型使用基于HDR片段的翻译规则。HDR片段是由中心词及其所有依存节点组成的树片段。这种翻译规则可以较好地捕捉语言中的句子模式和短语模式等组合现象,但在捕捉非组合现象(如习惯用语或固定搭配)方面存在不足。这类非组合现象易于由短语捕捉。为了更好地改善依存树到串模型的性能,本文提出了三种引入双语短语的方法,分别为引入句法短语、引入泛化句法短语及引入非句法短语。实验结果表明,同时使用句法短语、泛化句法短语及非句法短语时,可以将依存树到串模型的性能显著提高约1.0 BLEU值。     

《中国语言生活状况报告》中成语与习语的调查与思考

成语与习语的调查是《中国语言生活状况报告》在2007年的新增项目,这表明成语与习语使用情况引起了人们更多的关注。成语与习语的研究在语言应用中有广泛而深刻的意义。该文在基于大规模真实语料调查的基础之上,对成语与习语的使用情况做出了“单字差异”等比较,从中发现一些语言现象并提出了自己的思考,以期对汉语语言事实的发现、语言规律的总结、语言词汇的规范化等方面有所裨益。     

基于混合策略的英语基本名词短语识别——边界统计和词性串规则校正相结合的策略

基本名词短语识别是自然语言处理领域非常重要的子任务。文中总结了一些有代表性的基本名词短语识别方法,并对多种典型英语基本名词短语识别的结果进行了比较和对照,提出并实现了边界统计和词性串校正相结合的英语基本名词短语识别方法。该方法把基本名词短语识别分成主次分明的两部分,边界统计作为主要部分能够正确识别出大部分基本名词短语,词性串规则作为辅助手段在对前者识别出的基本名词短语进行核对和校正的同时还对边界统计方法遗漏的基本名词短语进行再回收。此方法中,词性串规则弥补了边界统计无法顾及基本名词短语内部组合规律的缺点,提高了精确率和召回率。采用此方法,基本名词短语识别的精确率达到96.22%,召回率97.59%,Fβ=196.90%,F值超出了目前报道的最好结果。     

基于短语替换的汉越伪平行句对生成

神经机器翻译在语料丰富的语种上取得了良好的翻译效果,但是在汉语-越南语这类双语资源稀缺的语种上性能不佳,通过对现有小规模双语语料进行词级替换生成伪平行句对可以较好地缓解此类问题。考虑到汉越词级替换中易存在一词多译问题,该文对基于更大粒度的替换进行了研究,提出了一种基于短语替换的汉越伪平行句对生成方法。利用小规模双语语料进行短语抽取构建短语对齐表,并通过在维基百科中抽取的实体词组对其进行扩充,在对双语数据的汉语和越南语分别进行短语识别后,利用短语对齐表中与识别出的短语相似性较高的短语对进行替换,以此实现短语级的数据增强,并将生成的伪平行句对与原始数据一起训练最终的神经机器翻译模型。在汉-越翻译任务上的实验结果表明,通过短语替换生成的伪平行句对可以有效提高汉-越神经机器翻译的性能。     

汉语篇章时间短语的分析与时制验算

汉英机器翻译中，汉语篇章的时间信息是生成正确英语词时态的基础，时制是时间信息重要组成部分，需要在篇间中通过时间短语的语义分析获得，首先对汉语篇章时间短时间了语义分类，然后设计了时间短语语义表示结构TPSRS，用概念信息体关联网络CIURN表示了汉语篇间语境知识，给出了在篇章语境中分析时间短语的算法TPPA，提出了通过时制验算来推导汉语篇章中时间短语的时制和事件的时制，最后在汉英机译系统ICENT中进行了实现，对已知写作时间的汉语篇章取得了较好的实验结果。     

An approach to automatic acquisition of translation templates based on phrase structure extraction and alignment

In this paper, we propose a new approach for automatically acquiring translation templates from unannotated bilingual spoken language corpora. Two basic algorithms are adopted: a grammar induction algorithm, and an alignment algorithm using bracketing transduction grammar. The approach is unsupervised, statistical, and data-driven, and employs no parsing procedure. The acquisition procedure consists of two steps. First, semantic groups and phrase structure groups are extracted from both the source language and the target language. Second, an alignment algorithm based on bracketing transduction grammar aligns the phrase structure groups. The aligned phrase structure groups are post-processed, yielding translation templates. Preliminary experimental results show that the algorithm is effective.