基于百科语料的中英文双语词典提取

双语词典是跨语言自然语言处理中一项非常重要的资源。目前提取双语词典的方法主要是基于平行语料库和基于可比语料库,但是这两种方法在提取新词或者某些技术术语时都存在双语资源匮乏的问题。相比之下,基于部分双语语料的方法由于利用的是新闻或者百科知识,故可以很好地解决这个问题,然而目前基于部分双语语料的方法主要集中在对文本内容的提取上,缺乏对文本内容以外部分的提取。针对此不足,该文以中英文两种语言为例,提出了一种基于百科语料的中英文双语词典的提取方法。该方法是在对文本内容提取的基础上结合在线百科的结构特点,分别用五种不同的方法对百科语料进行提取,综合查重后得到的双语信息数量为969 308条。与以往的基于部分双语语料的双语词典的提取方法相比,该方法在在线百科语料上的提取数量提高了170.75%。     

基于配价的维吾尔语框架语义知识库的构建

本文阐述了以配价作为基本描写法、真实语料为事实依据的维吾尔语框架语义知识库(简称框架网FrameNet)的构建,该知识库在构建维吾尔语词汇及其所属框架的语义词典等诸多领域有着广阔的应用空间和发展前景。提出了研究维吾尔语中句法功能和概念结构(也就是语义结构) 之间的关系, 以及建立用于自然语言处理的维吾尔语网上词汇知识库的意义。在维吾尔语的研究中引入了框架语义知识库(框架网)。框架语义知识库作为一种网上词汇语料库, 包括对每个词位( lexeme)的各个涵义的句法、语义信息的详尽描述。本文为维吾尔语框架语义知识库中各个框架元素的句法、语义特征的说明等自然语言信息处理研究提出新的研究思路,对基于配价的维吾尔语框架语义知识库构建的方法进行了探讨。     

The engineering of a Translator Workstation

The design of a Translator Workstation is an engineering task involving technical aspects like dictionary organization/retrieval and natural language processing, as well as human aspects like friendly user interface and user accessibility. This paper describes the design and implementation of such a workstation for translating English texts into Malay. Although specifically developed for English-Malay translation work, most of the ideas and features presented are language independent. Important issues raised include translation modes and functions, dictionary and thesaurus organization, morphological processing, spelling check and special word processing features.     

基于Hash结构词典的双向最大匹配分词法

针对当前自然语言处理中中文分词基于词典的机械分词方法,正序词典不能作为逆向最大匹配分词词典以及反序词典维护困难的问题,提出一种新的词典构造方法并设计了相应的双向最大匹配算法,同时在算法中加入了互信息歧义处理模块来处理分词中出现的交集型歧义。该算法可以在分词的过程中显著提高分词的精确度,适用于对词语切分精度要求较高的中文语言处理系统。     

网络用语词典的构建及问题分析

随着互联网应用的快速发展,网络用语的使用越来越普遍,网络新词层出不穷。网络文本中大量的网络用语,对基于自然语言处理的情感分析、产品推荐、问答系统等应用带来了一定的挑战,而收集并构建网络用语词典及相关语料则是解决此类问题的突破点。该文以微博语料为出发点,综合多类网络资源,收集并整理了较为全面的网络用语词典及相关语料。同时,对网络用语词典构建中遇到的问题进行了分析和总结,并对其潜在应用进行了初步的探讨。     

A method of creating new valency entries

Information on subcategorization and selectional restrictions in a valency dictionary is important for natural language processing tasks such as monolingual parsing, accurate rule-based machine translation and automatic summarization. In this paper we present an efficient method of assigning valency information and selectional restrictions to entries in a bilingual dictionary, based on information in an existing valency dictionary. The method is based on two assumptions: words with similar meaning have similar subcategorization frames and selectional restrictions; and words with the same translations have similar meanings. Based on these assumptions, new valency entries are constructed for words in a plain bilingual dictionary, using entries with similar source-language meaning and the same target-language translations. We evaluate the effects of various measures of semantic similarity.     

COMLEX Syntax – A Large Syntactic Dictionary for Natural Language Processing

This article is a detailed account of COMLEX Syntax, an on-line syntactic dictionary of English, developed by the Proteus Project at New York University under the auspices of the Linguistics Data Consortium. This lexicon was intended to be used for a variety of tasks in natural language processing by computer and as such has very detailed classes with a large number of syntactic features and complements for the major parts of speech and is, as far as possible, theory neutral. The dictionary was entered by hand with reference to hard copy dictionaries, an on-line concordance and native speakers‘intuition. Thus it is without prior encumbrances and can be used for both pure research and commercial purposes. This revised version was published online in August 2006 with corrections to the Cover Date.     

利用数据挖掘扩充量词名词词典的方法

研究名词与量词的关系对于自然语言处理有很大的帮助，目前用于自然语言处理用的量词名词词典是由人工方法实现的，并由人工进行维护，有更新慢、收藏词少等缺点。该文利用数据挖掘技术对大规模真实文本进行分析，挖掘量词与名词的深层关系，有效地实现了对人工量词名词词典的扩充。     

藏文语义词典建设中语义关系抽取方法的研究

研究语义是当前人工智能、语义网、语义词典等研究领域的热点,它可以有效支持机器翻译和自然语言处理等技术。文章根据藏文独特的文法特性,运用藏文逻辑格和计算语言学知识,在保留藏文原有特点的基础上,为藏文语义关系抽取方法建立较完整的语义场,以此为藏文语义词典建设提供了基础性构建方法。     

班智达藏文语料切分词典的建立与算法研究

随着自然语言信息处理的不断发展和完善,大规模语料文本处理已经成为计算语言学界的一个热门话题。一个重要的原因是从大规模的语料库中能够提取出所需要的知识。而语料文本的处理与加工以语法信息词典作基础。结合藏文语料库切分标注规范,论述了对藏文语料库切分与标注用的藏文语法信息词典的建立和设计,重点讨论了该词典的内容建设、语法信息的标注、索引结构及查找算法。     

面向小词典的高效英汉双语语料对齐算法

双语语料自动对齐是自然语言处理的一个重要研究课题。该文针对基于词典译文的英汉句子对齐算法存在的缺点，提出了面向小词典的高效英汉句子对齐算法，该算法在小词典的情况下仍具有较高的准确率，效率比传统算法提高近一倍。通过理论分析、对比实验可知，该算法是有效的。     

A full and efficient machine tractable dictionary for natural language processing: A revised version of the CUVOALD

A lexicon is an essential part of any natural language processing system. The size, content and format of the lexicon is crucial in determining the power and sophistication of a natural language processing system. However, a lexicon which provides comprehensive, consistent and accurate lexical information and which is in a format facilitating fast retrieval is not easily available. This paper reports on a project which aims at the development of such a lexicon. The resulting lexicon is actually the modified and extended version of the machine tractable version of the Oxford Advanced Learner's Dictionary. The modification and extension concentrate mainly on the aspects of comprehensiveness, consistency, explicitness, accuracy and the dictionary format. The modified and extended version is considered a desirable source of lexical information for any natural language processing system.Si-Qing Chen received her Ph.D. in computational linguistics (1992) from Guangzhou Institute of Foreign Languages. His research interests include natural language processing, computer-assisted language teaching, and computational lexicography. She is currently furthering her study and research at the University of Illinois at Urbana-Champaign. She has published several articles on psycholinguistics, language teaching, philosophy of language, and natural language processing in, amongst other journals, Language Learning and the CALICO Journal.Luomai Xu is an associate professor in the Department of English at Guangzhou Institute of Foreign Languages. He has been working on several projects and published several articles on natural language processing.     

基于HMM的哈萨克语词性标注研究

词性标注在自然语言信息处理领域中扮演着重要角色,是句法分析、信息抽取、机器翻译等自然语言处理的基础,对于哈萨克语同样如此。在基于词典静态标注的基础上分析了隐马尔科夫模型HMM(H idden M arkovModel)模型参数的选取、数据平滑以及未登录词的处理方法,利用基于统计的方法对哈萨克语熟语料进行训练,然后用V iterb i算法实现词性标注。实验结果表明利用HMM进行词性标注的准确率有所提高。     

一种基于修饰关系的自然语言语义分析方法

自然语言语义分析是自然语言处理技术走向深层应用的瓶颈。当前在概念、关系层次上的语义分析方法主要有两种:基于统计的特征向量抽取方法和基于语义词典(WordNet、HowNet等)的语义相似度计算方法。对于具体应用这两种方法都具有较大不足,前者由于统计模型的关系只适用于段落、篇章或多文档等粗粒度的语义分析,而不适合在句子词汇一级的应用;后者能方便处理实体概念之间的各种关系,但是如果想正确处理真实文本中的复杂修饰关系如概念与事件、概念与概念修饰、事件与事件修饰等关系,还需对语义词典和计算方法做进一步的扩展。提出了按照真实文本语句中词语之间修饰关系建立知识库,并设计了根据该知识库中已有修饰关系计算未知关系的算法;提出了可以依照修饰关系建立自然语言构句法的思路并给出了相关算法;最后给出了在语义分析系统上的实验,结果证明该方法是有效的。     

藏文自动分词系统的设计

语料库作为基本的语言数据库和知识库,是各种自然语言处理方法实现的基础。随着统计方法在自然语言处理中的广泛应用,语料库建设已成为重要的研究课题。自动分词是句法分析的一项不可或缺的基础性工作,其性能直接影响句法分析。本文通过对85万字节藏语语料的统计分析和藏语词的分布特点、语法功能研究,介绍基于词典库的藏文自动分词系统的模型,给出了切分用词典库的结构、格分块算法和还原算法。系统的研制为藏文输入法研究、藏文电子词典建设、藏文字词频统计、搜索引擎的设计和实现、机器翻译系统的开发、网络信息安全、藏文语料库建设以及藏语语义分析研究奠定了基础。     

基于词向量的藏文语义相似词知识库构建

词向量在自然语言处理研究的各个领域发挥着重要作用。该文从语言学角度出发,讨论了词向量技术与语言学理论的关系;根据词向量的特征,提出利用藏文词向量构建语义相似词知识库。该文以哈尔滨工业大学的《词林》为基础,通过汉藏双语词典对译,在获取对译词的词向量的基础上,计算对译词的词向量与原子词群平均词向量的差值,利用不同的差值,自动筛选出与原子词群语义相似度较小的词。该文分别以藏文的词和音节为单位计算词向量,自动筛出不属于原子词群的词,通过对自动筛选结果与人工筛选结果对比,发现两者具有较高的一致性,这说明词向量计算结果与人的语言直觉具有较高的一致性。总体来说,该文所采用的方法有助于提高藏文语义相似词知识库构建效率。     

Modifying the TEI DTD: The Case of Korean Dictionaries

Dictionary markup is one of the concerns of the Text Encoding Initiative (TEI), an international project for text encoding. In this paper, we investigate ways to use and extend the TEI encoding scheme for the markup of Korean dictionary entries. Since TEI suggestions for dictionary markup are mainly for western language dictionaries, we need to cope with problems to be encountered in encoding Korean dictionary entries. We try to extend and modify the TEI encoding scheme in the way suggested by the TEI. Also, we restrict the content model so that the encoded dictionary might be viewed as a database as well as a computerized, originally printed, dictionary. This revised version was published online in July 2006 with corrections to the Cover Date.     

实义词和相对位置的双语平行语料库自动构建

双语语料库自动对齐是自然语言处理的一个重要研究课题。针对基于词典的英汉双语句子对齐算法存在的缺点,提出了基于词典和相对位置高效英汉对齐算法,该算法在多数情况下具有较高的准确率,效率比传统算法好,通过理论的分析和实验对比可知,该算法是可行的。     

以词汇知识驱动的词网自动对映

机读字典蕴藏着非常丰富的词汇语意知识,这些知识可由自动化方式粹取出来,有效地利用在各种自然语言处理相关研究上。本研究提出一套方法,以英文版的WordNet 作为基本骨架,结合比对属类词与比对定义内容两种技巧,将WordNet同义词集对映到朗文当代英汉双语词典之词条。并藉由这个对映将WordNet同义词集冠上中文翻译词汇。在实验部分,我们依岐义程度将词汇分为单一语意与语意岐义两部分进行。在单一语意部分的实验结果,以100%的涵盖率计算,可获得97.7%的精准率。而在语意岐义部分,我们得到85.4%精准率,以及63.4%涵盖率的实验结果。     

Dictionary word sense distinctions: An enquiry into their nature

The word senses in a published dictionary are a valuable resource for natural language processing and textual criticism alike. In order that they can be further exploited, their nature must be better understood. Lexicographers have always had to decide where to say a word has one sense, where two. The two studies described here look into their grounds for making distinctions. The first develops a classification scheme to describe the commonly occurring distinction types. The second examines the task of matching the usages of a word from a corpus with the senses a dictionary provides. Finally, a view of the ontological status of dictionary word senses is presented.Adam Kilgarriff has recently completed his doctoral thesis, entitled Polysemy, available as CSRP 261, from the School of Cognitive and Computing Science, University of Sussex. He is now working on the preparation of database versions of dictionaries for language research for Longman Dictionaries.