Wrapper-filter feature selection algorithm using a memetic framework.

This correspondence presents a novel hybrid wrapper and filter feature selection algorithm for a classification problem using a memetic framework. It incorporates a filter ranking method in the traditional genetic algorithm to improve classification performance and accelerate the search in identifying the core feature subsets. Particularly, the method adds or deletes a feature from a candidate feature subset based on the univariate feature ranking information. This empirical study on commonly used data sets from the University of California, Irvine repository and microarray data sets shows that the proposed method outperforms existing methods in terms of classification accuracy, number of selected features, and computational efficiency. Furthermore, we investigate several major issues of memetic algorithm (MA) to identify a good balance between local search and genetic search so as to maximize search quality and efficiency in the hybrid filter and wrapper MA.     

基于文化基因算法和最小二乘支持向量机的安全数据特征处理方法

随着各类生物智能演化算法的日益成熟,基于演化技术及其混合算法的特征选择方法不断涌现。针对高维小样本安全数据的特征选择问题,将文化基因算法(Memetic Algorithm,MA)与最小二乘支持向量机(Least Squares Support Vector Machine,LS-SVM)进行结合,设计了一种封装式(Wrapper)特征选择方法(MA-LSSVM)。该方法利用最小二乘支持向量机易于求解的特点来构造分类器,以分类的准确率作为文化基因算法寻优过程中适应度函数的主要成分。实验表明,MA-LSSVM可以较高效地、稳定地获取对分类贡献较大的特征,降低了数据维度,提高了分类效率。     

A novel hybrid algorithm for feature selection

Feature selection is an important filtering method for data analysis, pattern classification, data mining, and so on. Feature selection reduces the number of features by removing irrelevant and redundant data. In this paper, we propose a hybrid filter–wrapper feature subset selection algorithm called the maximum Spearman minimum covariance cuckoo search (MSMCCS). First, based on Spearman and covariance, a filter algorithm is proposed called maximum Spearman minimum covariance (MSMC). Second, three parameters are proposed in MSMC to adjust the weights of the correlation and redundancy, improve the relevance of feature subsets, and reduce the redundancy. Third, in the improved cuckoo search algorithm, a weighted combination strategy is used to select candidate feature subsets, a crossover mutation concept is used to adjust the candidate feature subsets, and finally, the filtered features are selected into optimal feature subsets. Therefore, the MSMCCS combines the efficiency of filters with the greater accuracy of wrappers. Experimental results on eight common data sets from the University of California at Irvine Machine Learning Repository showed that the MSMCCS algorithm had better classification accuracy than the seven wrapper methods, the one filter method, and the two hybrid methods. Furthermore, the proposed algorithm achieved preferable performance on the Wilcoxon signed-rank test and the sensitivity–specificity test.     

A novel hybrid feature selection via Symmetrical Uncertainty ranking based local memetic search algorithm

A novel correlation based memetic framework (MA-C) which is a combination of genetic algorithm (GA) and local search (LS) using correlation based filter ranking is proposed in this paper. The local filter method used here fine-tunes the population of GA solutions by adding or deleting features based on Symmetrical Uncertainty (SU) measure. The focus here is on filter methods that are able to assess the goodness or ranking of the individual features. Empirical study of MA-C on several commonly used datasets from the large-scale Gene expression datasets indicates that it outperforms recent existing methods in the literature in terms of classification accuracy, selected feature size and efficiency. Further, we also investigate the balance between local and genetic search to maximize the search quality and efficiency of MA-C.     

Global geometric similarity scheme for feature selection in fault diagnosis

This work presents a global geometric similarity scheme (GGSS) for feature selection in fault diagnosis, which is composed of global geometric model and similarity metric. The global geometric model is formed to construct connections between disjoint clusters in fault diagnosis. The similarity metric of the global geometric model is applied to filter feature subsets. To evaluate the performance of GGSS, fault data from wind turbine test rig is collected, and condition classification is carried out with classifiers established by Support Vector Machine (SVM) and General Regression Neural Network (GRNN). The classification results are compared with feature ranking methods and feature wrapper approaches. GGSS achieves higher classification accuracy than the feature ranking methods, and better time efficiency than the feature wrapper approaches. The hybrid scheme, GGSS with wrapper, obtains optimal classification accuracy and time efficiency. The proposed scheme can be applied in feature selection to get better accuracy and efficiency in condition classification of fault diagnosis.     

一种基于信息增益及遗传算法的特征选择算法

特征选择是模式识别及数据挖掘等领域的重要问题之一。针对高维数据对象,特征选择一方面可以提高分类精度和效率,另一方面可以找出富含信息的特征子集。针对此问题,本文提出一种综合了filter模型及wrapper模型的特征选择方法,首先基于特征之间的信息增益进行特征分组及筛选,然后针对经过筛选而精简的特征子集采用遗传算法进行随机搜索,并采用感知器模型的分类错误率作为评价指标。实验结果表明,该算法可有效地找出具有较好的线性可分离性的特征子集,从而实现降维并提高分类精度。     

Normalized Mutual Information Feature Selection

A filter method of feature selection based on mutual information, called normalized mutual information feature selection (NMIFS), is presented. NMIFS is an enhancement over Battiti's MIFS, MIFS-U, and mRMR methods. The average normalized mutual information is proposed as a measure of redundancy among features. NMIFS outperformed MIFS, MIFS-U, and mRMR on several artificial and benchmark data sets without requiring a user-defined parameter. In addition, NMIFS is combined with a genetic algorithm to form a hybrid filter/wrapper method called GAMIFS. This includes an initialization procedure and a mutation operator based on NMIFS to speed up the convergence of the genetic algorithm. GAMIFS overcomes the limitations of incremental search algorithms that are unable to find dependencies between groups of features.      

Fast wrapper feature subset selection in high-dimensional datasets by means of filter re-ranking

This paper deals with the problem of supervised wrapper-based feature subset selection in datasets with a very large number of attributes. Recently the literature has contained numerous references to the use of hybrid selection algorithms: based on a filter ranking, they perform an incremental wrapper selection over that ranking. Though working fine, these methods still have their problems: (1) depending on the complexity of the wrapper search method, the number of wrapper evaluations can still be too large; and (2) they rely on a univariate ranking that does not take into account interaction between the variables already included in the selected subset and the remaining ones.Here we propose a new approach whose main goal is to drastically reduce the number of wrapper evaluations while maintaining good performance (e.g. accuracy and size of the obtained subset). To do this we propose an algorithm that iteratively alternates between filter ranking construction and wrapper feature subset selection (FSS). Thus, the FSS only uses the first block of ranked attributes and the ranking method uses the current selected subset in order to build a new ranking where this knowledge is considered. The algorithm terminates when no new attribute is selected in the last call to the FSS algorithm. The main advantage of this approach is that only a few blocks of variables are analyzed, and so the number of wrapper evaluations decreases drastically.The proposed method is tested over eleven high-dimensional datasets (2400-46,000 variables) using different classifiers. The results show an impressive reduction in the number of wrapper evaluations without degrading the quality of the obtained subset.     

基于相关性分析及遗传算法的高维数据特征选择

特征选择是模式识别及数据挖掘等领域的重要问题之一。针对高维数据对象,特征选择一方面可以提高分类精度和效率,另一方面可以找出富含信息的特征子集。针对此问题,提出了一种综合了filter模型及wrapper模型的特征选择方法,首先基于特征与类别标签的相关性分析进行特征筛选,只保留与类别标签具有较强相关性的特征,然后针对经过筛选而精简的特征子集采用遗传算法进行随机搜索,并采用感知器模型的分类错误率作为评价指标。实验结果表明,该算法可有效地找出具有较好的线性可分离性的特征子集,从而实现降维并提高分类精度。     

基于再生核希尔伯特空间映射的高维数据特征选择优化算法*

现有过滤型特征选择算法并未考虑非线性数据的内在结构,从而分类准确率远远低于封装型算法,对此提出一种基于再生核希尔伯特空间映射的高维数据特征选算法。首先,基于分支定界法建立搜索树,并对其进行搜索;然后,基于再生核希尔伯特空间映射分析非线性数据的内部结构;最终,根据数据集的内部结构选择最优的距离计算方法。对比仿真实验结果表明,本方法与封装型特征选择算法具有接近的分类准确率,同时在计算效率上具有明显的优势,适用于大数据分析。     

故障特征组合选择方法

特征选择方法主要包括过滤方法和绕封方法。为了利用过滤方法计算简单和绕封方法精度高的优点,提出一种组合过滤和绕封方法的特征选择新方法。该方法首先利用基于互信息准则的过滤方法得到满足一定精度要求的子集后,再采用绕封方法找到最后的优化特征子集。由于遗传算法在组合优化问题上的成功应用,对特征子集寻优采用了遗传算法。在数值仿真和轴承故障特征选择中,采用新方法在保证诊断精度的同时,可以节省大量选择时间。组合特征选择方法有较好的寻优特征子集的能力,能够节省选择时间,具有高效、高精度的双重优点。     

An effective feature selection method for hyperspectral image classification based on genetic algorithm and support vector machine

With the development and popularization of the remote-sensing imaging technology, there are more and more applications of hyperspectral image classification tasks, such as target detection and land cover investigation. It is a very challenging issue of urgent importance to select a minimal and effective subset from those mass of bands. This paper proposed a hybrid feature selection strategy based on genetic algorithm and support vector machine (GA–SVM), which formed a wrapper to search for the best combination of bands with higher classification accuracy. In addition, band grouping based on conditional mutual information between adjacent bands was utilized to counter for the high correlation between the bands and further reduced the computational cost of the genetic algorithm. During the post-processing phase, the branch and bound algorithm was employed to filter out those irrelevant band groups. Experimental results on two benchmark data sets have shown that the proposed approach is very competitive and effective.     

混合过滤器和封装器启发式判别籽棉成熟度

描述田间籽棉成熟度的形态结构和边界轮廓特征集存在维数灾难,其特征选择问题属于NP难题.基于交叉验证,提出了一种过滤器下浮动搜索并基于封装器停止搜索的求解算法.在训练集上以最大类可分性测量值为过滤器的评估函数启发式搜索最优l维特征子集(l=1,2,3,…),启发式规则包括最优特征组合和浮动搜索;在训练集上以Bayes分类器的误分率为封装器的评估函数对最优l维特征子集建模,模型在验证集上的平均误分率极小处产生的最优特征子集的容量为6,它们在预测集上的平均识别率为87.61%.在相关研究工作所涉及的40个数据集上验证算法的有效性,结果表明,在29个数据集上算法的分类性能好,执行效率高.     

An optimized extreme learning machine-based novel model for bearing fault classification

This work addresses the rolling element bearing (REB) fault classification problem by tackling the issue of identifying the appropriate parameters for the extreme learning machine (ELM) and enhancing its effectiveness. This study introduces a memetic algorithm (MA) to identify the optimal ELM parameter set for compact ELM architecture alongside better ELM performance. The goal of using MA is to investigate the promising solution space and systematically exploit the facts in the viable solution space. In the proposed method, the local search method is proposed along with link-based and node-based genetic operators to provide a tight ELM structure. A vibration data set simulated from the bearing of rotating machinery has been used to assess the performance of the optimized ELM with the REB fault categorization problem. The complexity involved in choosing a promising feature set is eliminated because the vibration data has been transformed into kurtograms to reflect the input of the model. The experimental results demonstrate that MA efficiently optimizes the ELM to improve the fault classification accuracy by around 99.0% and reduces the requirement of hidden nodes by 17.0% for both data sets. As a result, the proposed scheme is demonstrated to be a practically acceptable and well-organized solution that offers a compact ELM architecture in comparison to the state-of-the-art methods for the fault classification problem.     

Occupancy estimation from environmental parameters using wrapper and hybrid feature selection

Occupancy information is essential to facilitate demand-driven operations of air-conditioning and mechanical ventilation (ACMV) systems. Environmental sensors are increasingly being explored as cost effective and non-intrusive means to obtain the occupancy information. This requires the extraction and selection of useful features from the sensor data. In past works, feature selection has generally been implemented using filter-based approaches. In this work, we introduce the use of wrapper and hybrid feature selection for better occupancy estimation. To achieve a fast computation time, we introduce a ranking-based incremental search in our algorithms, which is more efficient than the exhaustive search used in past works. For wrapper feature selection, we propose the WRANK-ELM, which searches an ordered list of features using the extreme learning machine (ELM) classifier. For hybrid feature selection, we propose the RIG-ELM, which is a filter–wrapper hybrid that uses the relative information gain (RIG) criterion for feature ranking and the ELM for the incremental search. We present experimental results in an office space with a multi-sensory network to validate the proposed algorithms.     

Feature selection using correlation fractal dimension: Issues and applications in binary classification problems

Feature selection methods can be classified broadly into filter and wrapper approaches. Filter-based methods filter out features which are irrelevant to the target concept by ranking each feature according to some discrimination measure and then select features with high ranking value. In this paper, a filter-based feature selection method based on correlation fractal dimension (CFD) discrimination measure is proposed. One of the subgoals of this paper is to outline some issues that arise while calculating fractal dimension for higher dimensional ‘empirical’ data sets. It is well known that the calculation of fractal dimension for empirical data sets is meaningful only for an appropriate range of scales. We demonstrate through experimentation on data sets of various sizes that fractal dimension-based algorithms cannot be applied routinely to higher dimensional data sets as the calculation of fractal dimension is inherently sensitive to parameters like range of scales and the size of the data sets. Based on the empirical analysis, we propose a new feature selection technique using CFD that avoids the above issues. We successfully applied the proposed algorithm on a challenging classification problem in bioinformatics, namely, Promoter Recognition.     

mrPSO: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification

This paper presents a hybrid filter-wrapper feature subset selection algorithm based on particle swarm optimization (PSO) for support vector machine (SVM) classification. The filter model is based on the mutual information and is a composite measure of feature relevance and redundancy with respect to the feature subset selected. The wrapper model is a modified discrete PSO algorithm. This hybrid algorithm, called maximum relevance minimum redundancy PSO (mr²PSO), is novel in the sense that it uses the mutual information available from the filter model to weigh the bit selection probabilities in the discrete PSO. Hence, mr²PSO uniquely brings together the efficiency of filters and the greater accuracy of wrappers. The proposed algorithm is tested over several well-known benchmarking datasets. The performance of the proposed algorithm is also compared with a recent hybrid filter-wrapper algorithm based on a genetic algorithm and a wrapper algorithm based on PSO. The results show that the mr²PSO algorithm is competitive in terms of both classification accuracy and computational performance.     

Enhancing the performance of smart electrical grids using data mining and fuzzy inference engine

This paper is about enhancing the smart grid by proposing a new hybrid feature-selection method called feature selection-based ranking (FSBR). In general, feature selection is to exclude non-promising features out from the collected data at Fog. This could be achieved using filter methods, wrapper methods, or a hybrid. Our proposed method consists of two phases: filter and wrapper phases. In the filter phase, the whole data go through different ranking techniques (i.e., relative weight ranking, effectiveness ranking, and information gain ranking) The results of these ranks are sent to a fuzzy inference engine to generate the final ranks. In the wrapper phase, data is being selected based on the final ranks and passed on three different classifiers (i.e., Naive Bayes, Support Vector Machine, and neural network) to select the best set of the features based on the performance of the classifiers. This process can enhance the smart grid by reducing the amount of data being sent to the cloud, decreasing computation time, and decreasing data complexity. Thus, the FSBR methodology enables the user load forecasting (ULF) to take a fast decision, the fast reaction in short-term load forecasting, and to provide a high prediction accuracy. The authors explain the suggested approach via numerical examples. Two datasets are used in the applied experiments. The first dataset reported that the proposed method was compared with six other methods, and the proposed method was represented the best accuracy of 91%. The second data set, the generalization data set, reported 90% accuracy of the proposed method compared to fourteen different methods.

     

A novel ensemble-based wrapper method for feature selection using extreme learning machine and genetic algorithm

This paper presents a novel wrapper feature selection algorithm for classification problems, namely hybrid genetic algorithm (GA)- and extreme learning machine (ELM)-based feature selection algorithm (HGEFS). It utilizes GA to wrap ELM to search for the optimum subsets in the huge feature space, and then, a set of subsets are selected to make ensemble to improve the final prediction accuracy. To prevent GA from being trapped in the local optimum, we propose a novel and efficient mechanism specifically designed for feature selection problems to maintain GA’s diversity. To measure each subset’s quality fairly and efficiently, we adopt a modified ELM called error-minimized extreme learning machine (EM-ELM) which automatically determines an appropriate network architecture for each feature subsets. Moreover, EM-ELM has good generalization ability and extreme learning speed which allows us to perform wrapper feature selection processes in an affordable time. In other words, we simultaneously optimize feature subset and classifiers’ parameters. After finishing the search process of GA, to further promote the prediction accuracy and get a stable result, we select a set of EM-ELMs from the obtained population to make the final ensemble according to a specific ranking and selecting strategy. To verify the performance of HGEFS, empirical comparisons are carried out on different feature selection methods and HGEFS with benchmark datasets. The results reveal that HGEFS is a useful method for feature selection problems and always outperforms other algorithms in comparison.     

Ordinal-based and frequency-based integration of feature selection methods for sentiment analysis

Feature subset selection with the aim of reducing dependency of feature selection techniques and obtaining a high-quality minimal feature subset from a real-world domain is the main task of this research. For this end, firstly, two types of feature representation are presented for feature sets, namely unigram-based and part-of-speech based feature sets. Secondly, five methods of feature ranking are employed for creating feature vectors. Finally, we propose two methods for the integration feature vectors and feature subsets. An ordinal-based integration of different feature vectors (OIFV) is proposed in order to obtain a new feature vector. The new feature vector depends on the order of features in the old vectors. A frequency-based integration of different feature subsets (FIFS) with most effective features, which are obtained from a hybrid filter and wrapper methods in the feature selection task, is then proposed. In addition, four well-known text classification algorithms are employed as classifiers in the wrapper method for the selection of the feature subsets. A wide range of comparative experiments on five widely-used datasets in sentiment analysis were carried out. The experiments demonstrate that proposed methods can effectively improve the performance of sentiment classification. These results also show that proposed part-of-speech patterns are more effective in their classification accuracy compared to unigram-based features.