计算机自动分析量词短语的方法及规则

本文对现代汉语量词进行了分析和总结, 从组合关系的角度分析量词在句段中的地位, 根据量词与其它词的搭配关系总结出了有关分析量词短语的语法规则。在量词中, 有一些量词本身之间是同音词, 由于它们的语法功能相同, 所以单纯用语法上下文已无法区分它们了, 必须求助于语义上下文及不同量词与名词的搭配不同来分析它们, 本文利用量词与不同语义的名词的搭配关系来区分同音量词,这些方法及规则可应用于同音词的自动识别, 汉外机器翻译等方面, 我们已在微机止用C语言实现了同音量词自动识别系统。     

现代汉语名词语法属性的计量研究初探

以《现代汉语语法信息词典》中语法属性的概率化描述为目标,基于1998年上半年《人民日报》标注语料,对名词语法属性的概率化进行了初步的实验研究。首先,考察了名词与数词、名词与量词搭配的相关属性,引进“分散度”概念,利用它对“数名”结构进行了定量分析;其次,考察了名词受不同量词修饰的分布情况。最后,把实验结果与《现代汉语语法信息词典》的相应属性进行了比照和分析,在属性概率化的同时也对其正确性进行了验证。     

量词在知识图中的分类与表示

在当今知识表示领域中,知识图作为自然语言理解的语义模型有其独到之处,而在自然语言处理中普遍认为词是最基本的单位,本文从语义学和自然语言处理的角度（主要是从知识图的角度,）在对介词的逻辑词的研究之后,按照量词图的结构,对汉语中的量词进行了分类,并且按照知识量词构造,给一词图。     

中文基本复合名词短语语义关系体系及知识库构建

名词短语一直是中外语言学领域的重要研究对象,近年来在自然语言处理领域也受到了研究者的持续关注。英文方面,已建立了一定规模的名词短语语义关系知识库。但迄今为止,尚未建立相应或更大规模的描述名词短语语义关系的中文资源。该文借鉴国内外诸多学者对名词短语语义分类的研究成果,对大规模真实语料中的基本复合名词短语实例进行试标注与分析,建立了中文基本复合名词短语语义关系体系及相应句法语义知识库,该库能够为中文基本复合名词短语句法语义的研究提供基础数据资源。目前该库共含有18 281条高频基本复合名词短语,每条短语均标注了语义关系、短语结构及是否指称实体等信息,每条短语包含的两个名词还分别标注了语义类信息。语义类信息基于北京大学《现代汉语语义词典》。基于该知识库,该文还做了基本复合名词短语句法语义的初步统计与分析。     

基于量词的名词概念获取研究

概念获取是自然语言理解领域中重要的研究课题。该文提出了一种基于汉语量词的名词概念描述方法,设计并实现了一个权重计算方案。通过聚类实验探索了量词对名词语义区分的作用和贡献,实验结果表明基于量词的名词概念表达方式是有效的,可以区分大部分名词概念。     

汉语委婉语语言资源建设

委婉语是语言交流中不可或缺的交际手段,委婉语研究一直是语言学界的热门话题之一,但在自然语言处理领域,尚未有委婉语相关研究。该文借助现有纸质词典,基于语料库检索和专家人工判别的方式,初步构建了规模为63 000余条语料的汉语委婉语语言资源;并根据自然语言处理的相关任务需求,结合词典释义对委婉语进行分类。该文提出了利用同类委婉语的上下文语境辅助进行标注的方法。经过实验,对简单语义委婉语的语义判别准确率达89.71%,对语义复杂的兼类委婉语的语义判别准确率达74.65%,初步验证了利用计算机辅助人工标注构建委婉语语言资源的可行性。     

基于多策略的乌孜别克语名词词干识别研究

乌孜别克语名词词干识别是自然语言处理领域的基础研究,主要方法是从句子中提取名词词干,提高名词标注效率和准确性。该文首先陈述形态分析、形态特征对识别其词性的作用,然后讨论乌孜别克语的词类划分标准、名词的形态特征,乌孜别克语西尔里文转换拉丁文,乌孜别克语词汇翻译、标注技术,总结词缀歧义及消解规则。该文提出利用形态规则、词典、最大熵融合策略,设计现代乌孜别克语新词中名词词干识别算法,其中包括特征选择及参数估计、词内部特征、前后依存词特征等。最后以乌孜别克语网站文本作为验证对象,对名词词干进行统计与分析。     

无监督实体关系触发词词典自动构建

传统的实体关系触发词词典构建主要采用人工方法和有监督的扩展学习方法。但是,上述两种方法都需要大量的人工参与,并且当关系类型发生变化时需要重新构建触发词词典。提出一种无监督的实体关系触发词词典自动构建方法。首先,对关系实例文档集进行分层狄利克雷过程建模,通过主题过滤和词语概率权重过滤构建候选触发词集合;然后,利用依存句法分析对候选触发词集合进行再次过滤以得到最终的触发词词典。该方法有效避免了传统实体关系触发词词典构建所需的大量人工参与。实验表明,基于分层狄利克雷过程和依存句法分析的实体关系触发词词典自动构建方法有效降低了人工标注成本,取得了较高的准确率。     

基于Web数据的特定领域双语词典抽取

双语词典是跨语言检索以及机器翻译等自然语言处理应用的基础资源。本文提出了一种从非平行语料中抽取特定领域双语词典的算法。首先给出了算法的基本假设并回顾了相关的研究方法,然后详细给出了利用词间关系矩阵法从特定领域非平行语料中抽取双语词典的过程,最后通过大量实验分析了种子词选择对词典抽取结果的影响,实验结果表明种子词的数量和频率对词典抽取结果有积极作用。     

评价对象、短语、搭配关系抽取及倾向性判断

观点挖掘近年来已经成为自然语言处理领域的热点问题,该文对观点挖掘的几项关键技术—评价对象、评价短语、主观性关系抽取、倾向性判断进行了研究。在评价对象抽取阶段,通过统计得到所有的名词和名词短语作为候选,然后结合词频,词共现等特征进行过滤得到最终的评价对象;在评价短语抽取阶段,使用基于观点词词典的匹配方法,并把观点词前面的副词也作为评价短语的一部分;在搭配关系抽取阶段,目的是抽取评价对象和评价短语的关联关系,采取的方法是将在句中距离评级对象最近的评价短语作为该短语的评级短语;在情感倾向分析阶段,通过将情感句进行分类,然后制定规则进行无监督的倾向性判断。     

Sparse coding of i-vector/JFA latent vector over ensemble dictionaries for language identification systems

Computing sparse representation (SR) over an exemplar dictionary is time consuming and computationally expensive for large dictionary size. This also requires huge memory requirement for saving the dictionary. In order to reduce the latency and to achieve some diversity, ensemble of exemplar dictionary based language identification (LID) system is explored. The full diversity can be obtained if each of the exemplar dictionary contains only one feature vector from each of the language class. To achieve full diversity, a large number of multiple dictionaries are required; thus needs to compute SR for a particular test utterance as many times. The other solution to reduce the latency is to use a learned dictionary. The dictionary may contain unequal number of dictionary atoms and it is not guaranteed that each language class information is present. It totally depends upon the number of data and its variations. Motivated by this, language specific dictionary is learned, and then concatenated to form a single learned dictionary. Furthermore, to overcome the problem of ensemble exemplar dictionary based LID system, we investigated the ensemble of learned-exemplar dictionary based LID system. The proposed approach achieves the same diversity and latency as that of ensemble exemplar dictionary with reduced number of learned dictionaries. The proposed techniques are applied on two spoken utterance representations: the i-vector and the JFA latent vector. The experiments are performed on 2007 NIST LRE, 2009 NIST LRE and AP17-OLR datasets in closed set condition.     

SentiMI: Introducing point-wise mutual information with SentiWordNet to improve sentiment polarity detection

Supervised learning has attracted much attention in recent years. As a consequence, many of the state-of-the-art algorithms are domain dependent as they require a labeled training corpus to learn the domain features. This requires the availability of labeled corpora which is a cumbersome task in itself. However, for text sentiment detection SentiWordNet (SWN) may be used. It is a vocabulary where terms are arranged in synonym groups called synsets. This research makes use of SentiWordNet and treats it as the labeled corpus for training. A sentiment dictionary, SentiMI, builds upon the mutual information calculated from these terms. A complete framework is developed by using feature selection and extracting mutual information, from SentiMI, for the selected features. Training, testing and evaluation of the proposed framework are conducted on a large dataset of 50,000 movie reviews. A notable performance improvement of 7% in accuracy, 14% in specificity, and 8% in F-measure is achieved by the proposed framework as compared to the baseline SentiWordNet classifier. Comparison with the state-of-the-art classifiers is also performed on widely used Cornell Movie Review dataset which also proves the effectiveness of the proposed approach.     

联合过完备库下的信号稀疏分解

针对具有多个特征成分的复合信号在单一特征过完备库下无法实现稀疏分解的问题,提出构建联合过完备库的思想.联合过完备库由多个具有单一特征的过完备库联合构成,包含复合信号中各分量信号的信息,使得复合信号在其上具有稀疏性.利用稀疏分解算法,对多个复合信号在相应的联合过完备库上进行稀疏分解和信号重构,并与单一特征过完备库的分解结果进行了对比分析.仿真结果表明了构建联合过完备库思想的合理性和有效性.     

大规模中文实体情感知识的自动获取

目前中文情感分析的主要资源以情感词典为主,缺乏针对实体或属性的情感知识资源。该文主要研究如何从大规模文本语料中自动获取实体情感知识。在该文方法中,用情感表达组合来表示实体情感知识。首先,基于二部图排序算法对情感表达组合候选集合进行排序。然后,提出了一种基于语义相似的提炼算法对于排序靠后的表达组合进行选择。在提炼选择过程中,充分考虑实体之间和情感词之间的约束。最后,该文在三种大规模不同领域的语料上进行实验,并进行人工评价。评价结果表明,从三个领域数据集上获取的实体情感表达组合正确率均高于90%。最终我们获得了一个大规模情感知识词典,包括约30万对的情感表达组合。     

一种基于OCC模型的文本情感挖掘方法

观点挖掘（或情感分析）作为面向网络社会媒体分析挖掘领域的一个核心研究课题,具有重要的研究意义和应用价值。针对传统观点挖掘方法存在的不足和局限性,本文设计并实现了一种基于OCC情感模型的观点挖掘方法。该方法首先采用统计方法,利用WordNet词典、句法依存关系及少量标注数据,自动构建情感维度词典;其次,对所构建的情感维度词典进行求精,通过语义、情感倾向的不一致性处理和非情感词的过滤,得到高质量的情感维度词典;最后,基于所得到的情感维度词典,结合OCC模型中情感维度值与情感类型的对应关系,生成6种主要的情感类型。实验方法表明,此方法在使用灵活性、可解释性和有效性上具有明显的优势。     

基于编辑距离和多种后处理的生物实体名识别

基于编辑距离和多种后处理的生物医学文献实体名识别方法通过“全称缩写对识别算法”扩充词典,利用编辑距离算法提高识别召回率。在后处理阶段,使用前后缀词扩展、POS扩展、合并邻近实体及利用上下文线索等方法进一步提高性能。实验结果表明,使用该方法即使利用内部词典也可以获得较好的识别效果。     

SlangSD: building,expanding and using a sentiment dictionary of slang words for short-text sentiment classification

Sentiment information about social media posts is increasingly considered an important resource for customer segmentation, market understanding, and tackling other socio-economic issues. However, sentiment in social media is difficult to measure since user-generated content is usually short and informal. Although many traditional sentiment analysis methods have been proposed, identifying slang sentiment words remains a challenging task for practitioners. Though some slang words are available in existing sentiment lexicons, with new slang being generated with emerging memes, a dedicated lexicon will be useful for researchers and practitioners. To this end, we propose to build a slang sentiment dictionary to aid sentiment analysis. It is laborious and time-consuming to collect a comprehensive list of slang words and label the sentiment polarity. We present an approach to leverage web resources to construct a Slang Sentiment Dictionary (SlangSD) that is easy to expand. SlangSD is publicly available for research purposes. We empirically show the advantages of using SlangSD, the newly-built slang sentiment word dictionary for sentiment classification, and provide examples demonstrating its ease of use with a sentiment analysis system.     

维基百科中翻译对的模板挖掘方法研究

双语翻译对在跨语言信息检索、机器翻译等领域有着重要的用途,尤其是专有名词、新词、俚语和术语等的翻译是影响其系统性能的关键因素,但是这些翻译对很难从现有的词典中获得。该文针对维基百科的领域覆盖率和结构特征,提出了一种从维基百科中自动获取高质量中英文翻译对的模板挖掘方法,不但能有效地挖掘出常见的模板,而且能够发现人工不容易察觉的复杂模板。主要方法包括三步: 1)从语言工具栏中直接抽取翻译对,作为进一步挖掘的启发知识;2)在维基百科页面中采用PAT-Array结构挖掘中英翻译对模板;3)利用挖掘的模板在页面中自动挖掘其他中英文翻译对,并进行模板评估。实验结果表明,模板发现翻译对的正确率达90.4%。     

Discovery of Web Robot Sessions Based on their Navigational Patterns

Web robots are software programs that automatically traverse the hyperlink structure of the World Wide Web in order to locate and retrieve information. There are many reasons why it is important to identify visits by Web robots and distinguish them from other users. First of all, e-commerce retailers are particularly concerned about the unauthorized deployment of robots for gathering business intelligence at their Web sites. In addition, Web robots tend to consume considerable network bandwidth at the expense of other users. Sessions due to Web robots also make it more difficult to perform clickstream analysis effectively on the Web data. Conventional techniques for detecting Web robots are often based on identifying the IP address and user agent of the Web clients. While these techniques are applicable to many well-known robots, they may not be sufficient to detect camouflaged and previously unknown robots. In this paper, we propose an alternative approach that uses the navigational patterns in the click-stream data to determine if it is due to a robot. Experimental results on our Computer Science department Web server logs show that highly accurate classification models can be built using this approach. We also show that these models are able to discover many camouflaged and previously unidentified robots.     

Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification

The employed dictionary plays an important role in sparse representation or sparse coding based image reconstruction and classification, while learning dictionaries from the training data has led to state-of-the-art results in image classification tasks. However, many dictionary learning models exploit only the discriminative information in either the representation coefficients or the representation residual, which limits their performance. In this paper we present a novel dictionary learning method based on the Fisher discrimination criterion. A structured dictionary, whose atoms have correspondences to the subject class labels, is learned, with which not only the representation residual can be used to distinguish different classes, but also the representation coefficients have small within-class scatter and big between-class scatter. The classification scheme associated with the proposed Fisher discrimination dictionary learning (FDDL) model is consequently presented by exploiting the discriminative information in both the representation residual and the representation coefficients. The proposed FDDL model is extensively evaluated on various image datasets, and it shows superior performance to many state-of-the-art dictionary learning methods in a variety of classification tasks.