A fast SVM-based wrapper feature selection method driven by a fuzzy complementary criterion

The selection of informative and non-redundant features has become a prominent step in pattern classification. However, despite the intensive research, it is still an open issue to identify valuable feature subsets, especially in highly dimensional feature spaces. This paper proposes a wrapper feature selection method, in the context of support vector machines (SVMs), named Wr-SVM-FuzCoC. Our method combines effectively the advantages of the wrapper and filter approaches, achieving three goals simultaneously: classification performance, dimensionality reduction, and computational efficiency. In the filter part, a forward feature search methodology is developed, driven by a fuzzy complementary criterion, whereby at each iteration a feature is selected that exhibits the maximum additional contribution in regard to the previously selected subset. The quality of single features or feature subsets is assessed via a fuzzy local evaluation criterion with respect to patterns. This is achieved by the so-called fuzzy partition vector (FPV), comprising the fuzzy membership grades of every pattern in their target classes. Derivation of the feature FPVs is accomplished by incorporating a fuzzy output kernel-based support vector machine. The proposed method is favorably compared with existing SVM-based wrapper methods, in terms of performance capability and computational speed. Experimental investigation is carried out using a diverse pool of real datasets, including moderate and high-dimensional feature spaces.