情感词典自动构建方法综述

情感词典作为判断词语和文本情感倾向的重要工具, 其自动构建方法已成为情感分析和观点挖掘领域的一项重要研究内容. 本文整理了现有的中、英文情感词典资源, 同时分别从知识库、语料库、以及两者结合的角度, 归纳现有英文和中文情感词典的构建方法, 分析了各种方法的优缺点, 并总结了情感词典构建中的若干难点问题. 之后, 我们回顾了情感词典性能评估方法及相关评测竞赛. 最后总结了情感词典构建任务的发展前景以及一些亟需解决的问题.     

Building layered,multilingual sentiment lexicons at synset and lemma levels

Many tasks related to sentiment analysis rely on sentiment lexicons, lexical resources containing information about the emotional implications of words (e.g., sentiment orientation of words, positive or negative). In this work, we present an automatic method for building lemma-level sentiment lexicons, which has been applied to obtain lexicons for English, Spanish and other three official languages in Spain. Our lexicons are multi-layered, allowing applications to trade off between the amount of available words and the accuracy of the estimations. Our evaluations show high accuracy values in all cases. As a previous step to the lemma-level lexicons, we have built a synset-level lexicon for English similar to SentiWordNet 3.0, one of the most used sentiment lexicons nowadays. We have made several improvements in the original SentiWordNet 3.0 building method, reflecting significantly better estimations of positivity and negativity, according to our evaluations. The resource containing all the lexicons, ML-SentiCon, is publicly available.     

Senti‐CS: Building a lexical resource for sentiment analysis using subjective feature selection and normalized Chi‐Square‐based feature weight generation

Sentiment analysis involves the detection of sentiment content of text using natural language processing. Natural language processing is a very challenging task due to syntactic ambiguities, named entity recognition, use of slangs, jargons, sarcasm, abbreviations and contextual sensitivity. Sentiment analysis can be performed using supervised as well as unsupervised approaches. As the amount of data grows, unsupervised approaches become vital as they cut down on the learning time and the requirements for availability of a labelled dataset. Sentiment lexicons provide an easy application of unsupervised algorithms for text classification. SentiWordNet is a lexical resource widely employed by many researchers for sentiment analysis and polarity classification. However, the reported performance levels need improvement. The proposed research is focused on raising the performance of SentiWordNet3.0 by using it as a labelled corpus to build another sentiment lexicon, named Senti‐CS. The part of speech information, usage based ranks and sentiment scores are used to calculate Chi‐Square‐based feature weight for each unique subjective term/part‐of‐speech pair extracted from SentiWordNet3.0. This weight is then normalized in a range of ?1 to +1 using min–max normalization. Senti‐CS based sentiment analysis framework is presented and applied on a large dataset of 50000 movie reviews. These results are then compared with baseline SentiWordNet, Mutual Information and Information Gain techniques. State of the art comparison is performed for the Cornell movie review dataset. The analyses of results indicate that the proposed approach outperforms state‐of‐the‐art classifiers.     

基于领域情感词典特征表示的细粒度意见挖掘

细粒度意见挖掘的主要目标是从观点文本中获取情感要素并判断情感倾向。现有方法大多基于序列标注模型,但很少利用情感词典资源。该文提出一种基于领域情感词典特征表示的细粒度意见挖掘方法,使用领域情感词典在观点文本上构建特征表示并将其加入序列标注模型的输入部分。首先构建一份新的电商领域情感词典,然后在电商评论文本真实数据上,分别为条件随机场(CRF)和双向长短期记忆-条件随机场(BiLSTM-CRF)这两种常用序列标注模型设计基于领域情感词典的特征表示。实验结果表明,基于电商领域情感词典的特征表示方法在两种模型上都取得了良好的效果,并且超过其他情感词典。     

SentiMI: Introducing point-wise mutual information with SentiWordNet to improve sentiment polarity detection

Supervised learning has attracted much attention in recent years. As a consequence, many of the state-of-the-art algorithms are domain dependent as they require a labeled training corpus to learn the domain features. This requires the availability of labeled corpora which is a cumbersome task in itself. However, for text sentiment detection SentiWordNet (SWN) may be used. It is a vocabulary where terms are arranged in synonym groups called synsets. This research makes use of SentiWordNet and treats it as the labeled corpus for training. A sentiment dictionary, SentiMI, builds upon the mutual information calculated from these terms. A complete framework is developed by using feature selection and extracting mutual information, from SentiMI, for the selected features. Training, testing and evaluation of the proposed framework are conducted on a large dataset of 50,000 movie reviews. A notable performance improvement of 7% in accuracy, 14% in specificity, and 8% in F-measure is achieved by the proposed framework as compared to the baseline SentiWordNet classifier. Comparison with the state-of-the-art classifiers is also performed on widely used Cornell Movie Review dataset which also proves the effectiveness of the proposed approach.     

网络意见挖掘、摘要与检索研究综述

网络上带有人的主观感情色彩的评论性文本反映了人们的意见、态度和立场,因而具有很大的利用价值.信息挖掘技术针对这些主观文本进行处理,获得有用的意见、结论和知识.首先介绍了意见挖掘出现的背景和应用意义,然后从词汇情感极性识别、粗粒度的情感分类、细粒度的意见挖掘与摘要、意见检索和相关语言资源与系统5个方面综述了研究历程和现状,最后总结了研究难点与研究趋势.     

基于情感词典扩展技术的网络舆情倾向性分析

随着Web2.0时代的到来,网络已逐渐成为反映社会舆情的重要载体之一,网络舆情发现及网民的观点和倾向性挖掘也成为新的研究热点,但是目前尚无有效反应网民对热点事件或话题总体态度的舆情分析系统.本文针对网民关于话题评论简单、数目众多的特点,应用HowNet和NTUSD两种资源对现有情感词典进行扩展,建立了一个新的、具有倾向程度的情感词典.基于扩展的情感词典,开发了一个半自动化网络舆情分析系统.该系统能够为用户提供更加细致、准确的评论倾向性分析结果.     

Creating sentiment lexicon for sentiment analysis in Urdu: The case of a resource‐poor language

The sentiment analysis (SA) applications are becoming popular among the individuals and organizations for gathering and analysing user's sentiments about products, services, policies, and current affairs. Due to the availability of a wide range of English lexical resources, such as part‐of‐speech taggers, parsers, and polarity lexicons, development of sophisticated SA applications for the English language has attracted many researchers. Although there have been efforts for creating polarity lexicons in non‐English languages such as Urdu, they suffer from many deficiencies, such as lack of publically available sentiment lexicons with a proper scoring mechanism of opinion words and modifiers. In this work, we present a word‐level translation scheme for creating a first comprehensive Urdu polarity resource: “Urdu Lexicon” using a merger of existing resources: list of English opinion words, SentiWordNet, English–Urdu bilingual dictionary, and a collection of Urdu modifiers. We assign two polarity scores, positive and negative, to each Urdu opinion word. Moreover, modifiers are collected, classified, and tagged with proper polarity scores. We also perform an extrinsic evaluation in terms of subjectivity detection and sentiment classification, and the evaluation results show that the polarity scores assigned by this technique are more accurate than the baseline methods.     

Learning from context: A mutual reinforcement model for Chinese microblog opinion retrieval

This study addresses the problem of Chinese microblog opinion retrieval, which aims to retrieve opinionated Chinese microblog posts relevant to a target specified by a user query. Existing studies have shown that lexicon-based approaches employed online public sentiment resources to rank sentimentwords relying on the document features. However, this approach could not be effectively applied to microblogs that have typical user-generated content with valuable contextual information: “user–user” interpersonal interactions and “user–post/comment” intrapersonal interactions. This contextual information is very helpful in estimating the strength of sentiment words more accurately. In this study, we integrate the social contextual relationships among users, posts/comments, and sentiment words into a mutual reinforcement model and propose a unified three-layer heterogeneous graph, on which a random walk sentiment word weighting algorithm is presented to measure the strength of opinion of the sentiment words. Furthermore, the weights of sentiment words are incorporated into a lexicon-based model for Chinese microblog opinion retrieval. Comparative experiments are conducted on a Chinese microblog corpus, and the results show that our proposed mutual reinforcement model achieves significant improvement over previous methods.     

基于N-gram超核的中文倾向性句子识别

倾向性句子识别是文本倾向性分析的重要组成部分,其目的是识别文档中具有情感倾向的主观性句子。中文句子的倾向性不仅与倾向词有关,而且还跟句法、语义等因素有关,这使得倾向性句子识别不能简单地从词语的倾向性来统计得到。该文提出了一种基于N-gram超核的中文倾向性句子识别分类算法。该算法基于句子的句法、语义等特征构造N-gram超核函数,并采用基于该超核函数的支持向量机分类器识别中文倾向性句子。实验结果表明,与多项式核、N-gram核等单核函数相比,基于N-gram超核的中文倾向性句子识别算法在一定程度上能有效识别倾向性句子。     

基于双语信息和标签传播算法的中文情感词典构建方法

文本情感分析是目前自然语言处理领域的一个热点研究问题,具有广泛的实用价值和理论研究意义。情感词典构建则是文本情感分析的一项基础任务,即将词语按照情感倾向分为褒义、中性或者贬义。然而,中文情感词典构建存在两个主要问题 1)许多情感词存在多义、歧义的现象,即一个词语在不同语境中它的语义倾向也不尽相同,这给词语的情感计算带来困难;2)由国内外相关研究现状可知,中文情感字典建设的可用资源相对较少。考虑到英文情感分析研究中存在大量语料和词典,该文借助机器翻译系统,结合双语言资源的约束信息,利用标签传播算法(LP)计算词语的情感信息。在四个领域的实验结果显示我们的方法能获得一个分类精度高、覆盖领域语境的中文情感词典。     

Learning patterns for discovering domain-oriented opinion words

Sentiment analysis is a challenging task that attracted increasing interest during the last years. The availability of online data along with the business interest to keep up with consumer feedback generates a constant demand for online analysis of user-generated content. A key role to this task plays the utilization of domain-specific lexicons of opinion words that enables algorithms to classify short snippets of text into sentiment classes (positive, negative). This process is known as dictionary-based sentiment analysis. The related work tends to solve this lexicon identification problem by either exploiting a corpus and a thesaurus or by manually defining a set of patterns that will extract opinion words. In this work, we propose an unsupervised approach for discovering patterns that will extract domain-specific dictionary. Our approach (DidaxTo) utilizes opinion modifiers, sentiment consistency theories, polarity assignment graphs and pattern similarity metrics. The outcome is compared against lexicons extracted by the state-of-the-art approaches on a sentiment analysis task. Experiments on user reviews coming from a diverse set of products demonstrate the utility of the proposed method. An implementation of the proposed approach in an easy to use application for extracting opinion words from any domain and evaluate their quality is also presented.     

文本意见挖掘综述

意见挖掘是针对主观性文本自动获取有用的意见信息和知识,它是一个新颖而且十分重要的研究课题。这种技术可以应用于现实生活中的许多方面,如电子商务、商业智能、信息监控、民意调查、电子学习、报刊编辑、企业管理等。本文首先对意见挖掘进行了定义,然后阐述了意见挖掘研究的目的,接着从主题的识别、意见持有者的识别、陈述的选择和情感的分析四个方面对意见挖掘的研究现状进行了综述,并介绍了几个成型的系统。此外,我们针对汉语的意见挖掘做了特别的分析。最后对整个领域的研究进行了总结。     

Lexicon based semantic detection of sentiments using expected likelihood estimate smoothed odds ratio

Sentiment analysis is an active research area in today’s era due to the abundance of opinionated data present on online social networks. Semantic detection is a sub-category of sentiment analysis which deals with the identification of sentiment orientation in any text. Many sentiment applications rely on lexicons to supply features to a model. Various machine learning algorithms and sentiment lexicons have been proposed in research in order to improve sentiment categorization. Supervised machine learning algorithms and domain specific sentiment lexicons generally perform better as compared to the unsupervised or semi-supervised domain independent lexicon based approaches. The core hindrance in the application of supervised algorithms or domain specific sentiment lexicons is the unavailability of sentiment labeled training datasets for every domain. On the other hand, the performance of algorithms based on general purpose sentiment lexicons needs improvement. This research is focused on building a general purpose sentiment lexicon in a semi-supervised manner. The proposed lexicon defines word semantics based on Expected Likelihood Estimate Smoothed Odds Ratio that are then incorporated with supervised machine learning based model selection approach. A comprehensive performance comparison verifies the superiority of our proposed approach.     

领域相关的汉语情感词典扩展

动态情感知识的获取,特别是领域相关极性词典的构建一直是意见挖掘和情感分析系统在开放应用时面临的主要挑战之一。该文面向产品评价文本提出一种汉语情感极性词典扩展方法。该方法首先采用序列标注方法从意见文本中抽取产品意见要素,同时构建属性-评价对;然后,对抽取的属性-评价对进行正规化,以减少词典扩展中的复杂性和噪声;最后,改进PolarityRank算法的构图方式以使其适用于汉语文本,从而完成词典扩展。在汽车和手机两个领域的意见文本的实验结果表明领域相关的情感极性词语的扩展有利于情感极性分类性能的提高。
     

Exploiting semantic relationships for unsupervised expansion of sentiment lexicons

The literature in sentiment analysis has widely assumed that semantic relationships between words cannot be effectively exploited to produce satisfactory sentiment lexicon expansions. This assumption stems from the fact that words considered to be “close” in a semantic space (e.g., word embeddings) may present completely opposite polarities, which might suggest that sentiment information in such spaces is either too faint, or at least not readily exploitable. Our main contribution in this paper is a rigorous and robust challenge to this assumption: by proposing a set of theoretical hypotheses and corroborating them with strong experimental evidence, we demonstrate that semantic relationships can be effectively used for good lexicon expansion. Based on these results, our second contribution is a novel, simple, and yet effective lexicon-expansion strategy based on semantic relationships extracted from word embeddings. This strategy is able to substantially enhance the lexicons, whilst overcoming the major problem of lexicon coverage. We present an extensive experimental evaluation of sentence-level sentiment analysis, comparing our approach to sixteen state-of-the-art (SOTA) lexicon-based and five lexicon expansion methods, over twenty datasets. Results show that in the vast majority of cases our approach outperforms the alternatives, achieving coverage of almost 100% and gains of about 26% against the best baselines. Moreover, our unsupervised approach performed competitively against SOTA supervised sentiment analysis methods, mainly in scenarios with scarce information. Finally, in a cross-dataset comparison, our approach turned out to be as competitive as (i.e., statistically tie with) state-of-the-art supervised solutions such as pre-trained transformers (BERT), even without relying on any training (labeled) data. Indeed in small datasets or in datasets with scarce information (short messages), our solution outperformed the supervised ones by large margins.     

Hybrid sentiment analysis framework for a morphologically rich language

This paper presents a process of building a Sentiment Analysis Framework for Serbian (SAFOS). We created a hybrid method that uses a sentiment lexicon and Serbian WordNet (SWN) synsets assigned with sentiment polarity scores in the process of feature selection. As the use of stemming for morphologically rich languages (MRLs) may result in loss or giving incorrect sentiment meaning to words, we decided to expand the sentiment lexicon, as well as the lexicon generated using SWN, by adding morphological forms of emotional terms and phrases. It was done using Serbian Morphological Electronic Dictionaries. A new feature reduction method for document-level sentiment polarity classification using maximum entropy modeling is proposed. It is based on mapping of a large number of related feature candidates (sentiment words, phrases and their inflectional forms) to a few concepts and using them as features. Testing was performed on a 10-fold cross validation set and on test sets containing news and movie reviews. The results of all experiments show that sentiment feature mapping for feature set reduction achieves better results over the basic set of features. For both test sets, the best classification accuracy scores were achieved for the combination of unigram and bigram features reduced by sentiment feature mapping (accuracy 78.3 % for movie reviews and 79.2 % for news test set). In 10-fold cross-validation, best average accuracy score of 95.6 % was obtained using unigrams as features, reduced by the mapping procedure.     

Multiaspect‐based opinion classification model for tourist reviews

Tourist reviews on social media websites reflect the tourist's opinions concerning various aspects of a tourist place or service (e.g., “comfortable room” and “terrible service” in hotel reviews). Extracting these aspects from reviews is a challenging task in opinion mining. Therefore, aspect‐based opinion mining has emerged as a new area of social review mining. Existing approaches in this area focus on extracting explicit aspects and classification of opinions around these aspects. However, the implicit and coreferential aspects during aspect extraction are often neglected, and the classification of multiaspect opinions is relatively less emphasized in prior art. In this paper, we propose a model, namely, “enhanced multiaspect‐based opinion classification” that addresses existing challenges by automatically extracting both explicit and implicit aspects and classifying the multiaspect opinions. In this model, first, a probabilistic co‐occurrence‐based method is proposed that utilizes the co‐occurrence between aspects and sentiment words to identify the coreferential aspects and merge them into groups. Second, an implicit aspect extraction method is proposed that associates the sentiment words with suitable aspects to build an aspect‐sentiment hierarchy. Third, a multiaspect opinion classification approach is proposed that employs multilabel classification algorithms to classify opinions into different polarity classes. The effectiveness of the proposed model is evaluated by conducting experiments on benchmark and real‐world datasets. The experimental results revealed the supremacy of multilabel classifiers by achieving 90% accuracy per label on classification when extracting 87% domain‐relevant aspects. A state‐of‐the‐art performance comparison is conducted that also verifies the advantages of the proposed model.     

情感分析研究综述

由于Web文本迅速增多,对这些文本,特别是用户主动发布的评论数据进行挖掘和分析,识别出其情感趋向及演化规律,可以更好地理解用户的消费习惯,分析热点舆情,给企业、政府等机构提供重要的决策依据。首先对情感分析的研究对象和目标进行了定义和说明,并给出基本的研究思路。然后,在主观性句子识别任务上,详细回顾和分析了主要的处理方法;在观点分类的特征抽取上,重点介绍和讨论了两类主流的处理思路--基于情感词和基于频繁模式挖掘。接着简要介绍了其他一些相关的情感分析问题。最后总结了情感分析的现有成就和不足,以及面临的挑战,并对其发展前景进行了展望。     

基于情感从属和最大熵模型的细粒度观点挖掘

目前许多观点挖掘方法挖掘粒度过大,导致反馈信息不足。为解决该问题,对标准LDA模型进行改进,提出主题情感联合最大熵LDA模型进行细粒度观点挖掘。首先,考虑到词的位置和语义信息,在传统LDA模型中加入最大熵组件来区分背景词、特征词和观点词,并对特征词和观点词进行局部和全局的划分;其次,在主题层和单词层之间加入情感层,实现词语级别的细粒度情感分析,并引入情感转移变量来处理情感从属关系,同时获取整篇评论和每个主题的情感极性,实验验证了所提模型和理论的有效性。