一种新的基于信息熵的属性选择算法

属性选择是提高分类器性能的一种有效的方法.然而已有的属性选择算法要么假设数据无噪声,要么没有考虑属性间的交互作用,不能用于数据集中既有噪声又存在属性交互作用的情况.提出一种基于信息熵的属性选择算法,该算法用条件熵来评价属性子集对目标概念的描述能力,利用后向删除搜索策略进行属性选择.同时,根据不一致实例和关联规则中提升度的概念,给出噪声数据的定义和识别方法.该算法和典型的属性选择算法在10个UCI标准数据集上的对比实验结果表明,提出的算法在减少属性数量的同时将C4.5和NaiveBayes的平均分类精度分别提高了2.77%和3.42%.

A New Entropy-Based Feature Subset Selection Algorithm

Feature subset selection is a quite hot topic in the classifier research,and a large number of algorithms have already been proposed.However,none of them can handle both the feature interaction and the noise.In this paper,a new entropy-based feature subset algorithm is presented.In the proposed algorithm,conditional entropy is adopted to measure feature subset's discriminating ability for differentiating class labels.Backward elimination search is used to find the best subset.Meanwhile,the proposed algorithm is able to handle noise data.Experimental results show that the algorithm can address the feature interaction and further improve the performance of C4.5 and NaiveBayes.