基于假设检验的文本分类特征选择

在T-C(term-category)双向四格表中,特征与文档类相互独立与它们互不相关是等价的.基于此,本文应用了两种新颖的独立性假设检验方法来度量特征与文档类的相关程度,从文本集特征空间中选择能够高度代表文档内容的特征子集用于文本分类.实验结果表明,把假设检验应用于文本分类特征选择中,有利于提高分类性能.

Hypothesis Test-based Feature Selection for Text Categorization

For the feature and the document category from a T-C(term-category) two-way four-fold contingency table, their mutual independence is equivalent to their mutual non-correlation.At this point,this paper uses two novel hypothesis test methods of independence to measure the degree of correlation between features and categories,and accordingly the high representative feature subset of the document content is selected out of the feature space of the text set for text categorization. The results of experiments sh...