文本分类中影响因素的定量分析

基于包含全部特征的类别特征数据库,利用基于距离度量的Rocchio算法、Fast TC算法和基于概率模型的NB算法,从定量的角度来分析停用词、词干合并、数字和测试文档长度4个因素对文本分类精度的影响程度。实验表明,过滤停用词方法是一种无损的特征压缩手段,词干合并虽然对分类精度略有减弱,但仍能保证特征压缩的可行性。数字与其他词汇的语义关联性提高了Rocchio算法和Fast TC算法的分类精度,但降低了视特征彼此独立的NB算法的分类精度。3种算法在测试文档取不同数量的关键词时分类精度的变化趋势说明了特征所包含的有益信息和噪音信息对分类精度的影响。     

基于多特征选择的中文文本分类

自动文本分类就是在给定的分类体系下,让计算机根据文本的内容确定与它相关联的类别。特征选择作为文本分类中的关键,困难之一是特征空间的高维性,因此寻求一种有效的特征选择方法,降低特征空间的维数,成为文本分类中的重要问题。在分析已有的文本分类特征选择方法的基础上,实现了一种组合不同特征选择方法的多特征选择方法,应用于KNN文本分类算法,实验表明,多特征选择方法分类效果比单一的特征选择方法分类效果有明显的提高。     

Ensemble of keyword extraction methods and classifiers in text classification

Automatic keyword extraction is an important research direction in text mining, natural language processing and information retrieval. Keyword extraction enables us to represent text documents in a condensed way. The compact representation of documents can be helpful in several applications, such as automatic indexing, automatic summarization, automatic classification, clustering and filtering. For instance, text classification is a domain with high dimensional feature space challenge. Hence, extracting the most important/relevant words about the content of the document and using these keywords as the features can be extremely useful. In this regard, this study examines the predictive performance of five statistical keyword extraction methods (most frequent measure based keyword extraction, term frequency-inverse sentence frequency based keyword extraction, co-occurrence statistical information based keyword extraction, eccentricity-based keyword extraction and TextRank algorithm) on classification algorithms and ensemble methods for scientific text document classification (categorization). In the study, a comprehensive study of comparing base learning algorithms (Naïve Bayes, support vector machines, logistic regression and Random Forest) with five widely utilized ensemble methods (AdaBoost, Bagging, Dagging, Random Subspace and Majority Voting) is conducted. To the best of our knowledge, this is the first empirical analysis, which evaluates the effectiveness of statistical keyword extraction methods in conjunction with ensemble learning algorithms. The classification schemes are compared in terms of classification accuracy, F-measure and area under curve values. To validate the empirical analysis, two-way ANOVA test is employed. The experimental analysis indicates that Bagging ensemble of Random Forest with the most-frequent based keyword extraction method yields promising results for text classification. For ACM document collection, the highest average predictive performance (93.80%) is obtained with the utilization of the most frequent based keyword extraction method with Bagging ensemble of Random Forest algorithm. In general, Bagging and Random Subspace ensembles of Random Forest yield promising results. The empirical analysis indicates that the utilization of keyword-based representation of text documents in conjunction with ensemble learning can enhance the predictive performance and scalability of text classification schemes, which is of practical importance in the application fields of text classification.     

Text feature selection with a robust weight scheme and dynamic dimension reduction to text document clustering

This paper proposes three feature selection algorithms with feature weight scheme and dynamic dimension reduction for the text document clustering problem. Text document clustering is a new trend in text mining; in this process, text documents are separated into several coherent clusters according to carefully selected informative features by using proper evaluation function, which usually depends on term frequency. Informative features in each document are selected using feature selection methods. Genetic algorithm (GA), harmony search (HS) algorithm, and particle swarm optimization (PSO) algorithm are the most successful feature selection methods established using a novel weighting scheme, namely, length feature weight (LFW), which depends on term frequency and appearance of features in other documents. A new dynamic dimension reduction (DDR) method is also provided to reduce the number of features used in clustering and thus improve the performance of the algorithms. Finally, k-mean, which is a popular clustering method, is used to cluster the set of text documents based on the terms (or features) obtained by dynamic reduction. Seven text mining benchmark text datasets of different sizes and complexities are evaluated. Analysis with k-mean shows that particle swarm optimization with length feature weight and dynamic reduction produces the optimal outcomes for almost all datasets tested. This paper provides new alternatives for text mining community to cluster text documents by using cohesive and informative features.     

Comparison of text feature selection policies and using an adaptive framework

Text categorization is the task of automatically assigning unlabeled text documents to some predefined category labels by means of an induction algorithm. Since the data in text categorization are high-dimensional, often feature selection is used for reducing the dimensionality. In this paper, we make an evaluation and comparison of the feature selection policies used in text categorization by employing some of the popular feature selection metrics. For the experiments, we use datasets which vary in size, complexity, and skewness. We use support vector machine as the classifier and tf-idf weighting for weighting the terms. In addition to the evaluation of the policies, we propose new feature selection metrics which show high success rates especially with low number of keywords. These metrics are two-sided local metrics and are based on the difference of the distributions of a term in the documents belonging to a class and in the documents not belonging to that class. Moreover, we propose a keyword selection framework called adaptive keyword selection. It is based on selecting different number of terms for each class and it shows significant improvement on skewed datasets that have a limited number of training instances for some of the classes.     

基于类信息的文本特征选择与加权算法研究

文本自动分类中特征选择和加权的目的是为了降低文本特征空间维数、去除噪音和提高分类精度。传统的特征选择方案筛选出的特征往往偏爱类分布不均匀文档集中的大类,而常用的TF·IDF特征加权方案仅考虑了特征与文档的关系,缺乏对特征与类别关系的考虑。针对上述问题,提出了基于类别信息的特征选择与加权方法,在两个不同的语料集上进行比较和分析实验,结果显示基于类别信息的特征选择与加权方法比传统方法在处理类分布不均匀的文档集时能有效提高分类精度,并且降维程度有所提高。     

自动文本分类特征选择方法研究

文本分类是指根据文本的内容将大量的文本归到一个或多个类别的过程,文本表示技术是文本分类的核心技术之一,而特征选择又是文本表示技术的关键技术之一,对分类效果至关重要。文本特征选择是最大程度地识别和去除冗余信息,提高训练数据集质量的过程。对文本分类的特征选择方法,包括信息增益、互信息、X^2统计量、文档频率、低损降维和频率差法等做了详细介绍、分析、比较研究。     

A two-stage feature selection method for text categorization

Feature selection for text categorization is a well-studied problem and its goal is to improve the effectiveness of categorization, or the efficiency of computation, or both. The system of text categorization based on traditional term-matching is used to represent the vector space model as a document; however, it needs a high dimensional space to represent the document, and does not take into account the semantic relationship between terms, which leads to a poor categorization accuracy. The latent semantic indexing method can overcome this problem by using statistically derived conceptual indices to replace the individual terms. With the purpose of improving the accuracy and efficiency of categorization, in this paper we propose a two-stage feature selection method. Firstly, we apply a novel feature selection method to reduce the dimension of terms; and then we construct a new semantic space, between terms, based on the latent semantic indexing method. Through some applications involving the spam database categorization, we find that our two-stage feature selection method performs better.     

中文文本分类中特征选择方法的比较

在自动文本分类系统中,特征选择是有效的降维数方法.通过实验对中文文本分类中的特征选择方法逐一进行测试研究,力图确定较优的中文文本分类特征选择方法.根据实验得出:在所测试的所有特征选择方法中,统计方法的分类性能最好,其次为信息增益(IG),交叉熵(CE)和文本证据权(WE)也取得了较好的效果,互信息(MI)较差.     

使用特征分辨率和差别对象对集的特征选择

特征选择是文本分类的关键步骤之一,所选特征子集的优劣直接影响文本分类的结果。首先简单分析了几种经典的特征选择方法,总结了它们的不足,然后提出了特征分辨率的概念,并提出了一个基于差别对象对集的属性约简算法,最后把该属性约简算法同特征分辨率结合起来,提出了一个新的特征选择方法。该方法首先利用特征分辨率进行特征初选以过滤掉一些词条来降低特征空间的稀疏性,然后利用所提属性约简算法消除冗余,从而获得较具代表性的特征子集。实验结果表明此种特征选择方法效果良好。     

使用类内集中度和分层递阶约简的特征选择方法

特征选择是文本分类的关键步骤之一,所选特征子集的优劣直接影响文本分类的结果。首先简单分析了几种经典的特征选择方法,总结了它们的不足,然后提出了类内集中度的概念,紧接着把分层递阶的思想引入粗糙集并提出了一个改进的基于分层递阶的属性约简算法,最后把该约简算法同类内集中度结合起来,提出了一个综合的特征选择方法。该方法首先利用类内集中度进行特征初选以过滤掉一些词条来降低特征空间的稀疏性,然后利用所提约简算法消除冗余,从而获得较具代表性的特征子集。实验结果表明此种特征选择方法效果良好。     

基于类别分布的特征选择框架

目前已有很多种特征选择方法,但就目前所知,没有一种方法能够在非平衡语料上取得很好的效果.依据特征在类别间的分布特点提出了基于类别分布的特征选择框架.该框架能够利用特征的分布信息选出具有较强区分能力的特征,同时允许给类别灵活地分配权重,分配较大的权重给稀有类别则提高稀有类别的分类效果,所以它适用于非平衡语料,也具有很好的扩展性.另外,OCFS和基于类别分布差异的特征过滤可以看作该框架的特例.实现该框架得到了具体的特征选择方法,Retuers-21578语料及复旦大学语料等两个非平衡语料上的实验表明,它们的Macro 和Micro F1效果都优于IG, CHI和OCFS.     

一种基于词义和词频的向量空间模型改进方法

向量空间模型(VSM)是一种使用特征向量对文本进行建模的方法,广泛应用于文本分类、模式识别等领域。但文本内容较多时,传统的VSM建模可能产生维数爆炸现象,效率低下且难以保证分类效果。针对VSM高维现象,提出一种利用词义和词频降低文本建模维度的方法,以提高效率和准确度。提出一种多义词判别优化的同义词聚类方法,结合上下文判别多义词的词义后,根据特征项词义相似度进行加权,合并词义相近的特征项。新方法使特征向量维度大大降低,多义词判别提高了文章特征提取的准确性。与其他文本特征提取和文本分类方法进行比较,结果表明,该算法在效率和准确度上有明显提高。     

新的文本分类特征选择方法研究

特征降维是文本分类过程中的一个重要环节。在现有特征选择方法的基础上,综合考虑特征词在正类和负类中的分布性质,综合四种衡量特征类别区分能力的指标,提出了一个新的特征选择方法,即综合比率(CR)方法。实验采用K-最近邻分类算法(KNN)来考查CR方法的有效性,实验结果表明该方法能够取得比现有特征选择方法更优的降维效果。     

基于信息熵理论的特征权重算法研究

文本表示是使用分类算法处理文本时必不可少的环节,文本表示方法的选择对最终的分类精度起着至关重要的作用。针对经典的特征权重计算方法TFIDF（Term Frequency and Inverted Document Frequency）中存在的不足,提出了一种基于信息熵理论的特征权重算法ETFIDF（Entropy based TFIDF）。ETFIDF不仅考虑特征项在文档中出现的频率及该特征项在训练集中的集中度,而且还考虑该特征项在各个类别中的分散度。实验结果表明,采用ETFIDF计算特征权重可以有效地提高文本分类性能,对ETFIDF与特征选择的关系进行了较详细的理论分析和实验研究。实验结果表明,在文本表示阶段考虑特征与类别的关系可以更为准确地表示文本;如果综合考虑精度与效率两个方面因素,ETFIDF算法与特征选择算法一起采用能够得到更好的分类效果。     

基于交叉覆盖算法的中文文本分类

基于向量空间模型的文本分类过程中遇到的最大问题就是以词为特征项的向量维数太大，需要进行特征选取，而交叉覆盖算法的输入集是n维欧式空间的点集，可以忽略维数的大小，从而最大程度上精确地表示文本，然后再进行分类，能够大大提高正确率。将交叉覆盖算法作为一种分类算法来进行中文文本分类，取得了不错的效果，在封闭测试中的准确率达到98.32%。     

基于粗糙集的特征选择方法的研究

文本自动分类是指将文本按照一定的策略归于一个或多个类别中的应用技术。文本分类是文本挖掘的基础,而特征选择又是文本分类中的核心。论文分析了以前特征选择方法中由于特征数目过多而造成分类时间和精度不高的缺点,提出了一种基于粗糙集的特征选择方法,其特点是以特征在文本分类中的重要性对特征进行选择。最后通过实验验证了该算法,证明该方法是可行的。     

一种基于权重的文本特征选择方法

在分析比较几种常用的特征选择方法的基础上,提出了一种引入文本类区分加权频率的特征选择方法TFIDF_Ci。它将具体类的文档出现频率引入TFIDF函数,提高了特征项所在文档所属类区分其他类的能力。实验中采用KNN分类算法对该方法和其他特征选择方法进行了比较测试。结果表明,TFIDF_Ci方法较其他方法在不同的训练集规模情况下具有更高的分类精度和稳定性。     

文本分类中特征提取方法的比较与研究

特征的提取是文本分类中的关键技术,该文着重介绍了文本特征提取方法中的文档频率(DF)、信息增益(IG)、x2统计(CHI),该文通过实验对于以上三种特征提取方法进行了研究和比较,该结论对于今后研究特征选择方法对于英文作文自动评分的影响奠定了理论和实践基础。