A hybrid multiple feature construction approach for classification using Genetic Programming

The purpose of feature construction is to create new higher-level features from original ones. Genetic Programming (GP) was usually employed to perform feature construction tasks due to its flexible representation. Filter-based approach and wrapper-based approach are two commonly used feature construction approaches according to their different evaluation functions. In this paper, we propose a hybrid feature construction approach using genetic programming (Hybrid-GPFC) that combines filter’s fitness function and wrapper’s fitness function, and propose a multiple feature construction method that stores top excellent individuals during a single GP run. Experiments on ten datasets show that our proposed multiple feature construction method (Fcm) can achieve better (or equivalent) classification performance than the single feature construction method (Fcs), and our Hybrid-GPFC can obtain better classification performance than filter-based feature construction approaches (Filter-GPFC) and wrapper-based feature construction approaches (Wrapper-GPFC) in most cases. Further investigations on combinations of constructed features and original features show that constructed features augmented with original features do not improve the classification performance comparing with constructed features only. The comparisons with three state-of-art methods show that in majority of cases, our proposed hybrid multiple feature construction approach can achieve better classification performance.