基于区分类别能力的高性能特征选择方法

特征选择在文本分类中起着重要作用.文档频率(document frequency,简称DF)、信息增益(informationgain,简称IG)和互信息(mutualin formation,简称MI)等特征选择方法在文本分类中广泛应用.已有的实验结果表明,IG是最有效的特征选择算法之一,DF稍差,而MI效果相对较差.在文本分类中,现有的特征选择函数性能的评估均是通过实验验证的方法,即完全是基于经验的方法.特征选择是选择部分最有区分类别能力的特征,为此,给出了两个特征选择函数需满足的基本约束条件,并提出了一种构造高性能特征选择的通用方法.依此方法构造了一个新的特征选择函数KG(knowledge gain).分析发现,IG和KG完全满足该构造方法,在Reuters-21578,OHSUMED和News Group这3个语料集上的实验表明,IG和KG性能最好,在两个语料集上,KG甚至超过了IG.验证了提出的构造高性能特征选择函数方法的有效性,同时也在理论上给出了一个评价高性能特征选择算法的标准.

A Category Resolve Power-Based Feature Selection Method

One of the most important issues in Text Categorization (TC) is Feature Selection (FS). Many FS methods have been put forward and widely used in TC field, such as Information Gain (IG), Document Frequency (DF) thresholding, Mutual Information (MI) and so on. Empirical studies show that IG is one of the most effective methods, DF performs similarly, in contrast, and MI had relatively poor performance. One basic research question is why these FS methods cause different performance. Many existing work answers this question based on empirical studies. This paper presents a formal study of FS based on category resolve power. First, two desirable constraints that any reasonable FS function should satisfy are defined, then a universal method for developing FS functions is presented, and a new FS function KG using this method is developed. Analysis shows that IG and KG (knowledge gain) satisfy this universal method. Experiments on Reuters-21578 collection, NewsGroup collection and OHSUMED collection show that KG and IG get the best performance, even KG performs better than the IG method in two collections. These experiments imply that the universal method is very effective and gives a formal evaluation criterion for FS method.