Simultaneous feature selection and classification using kernel-penalized support vector machines

We introduce an embedded method that simultaneously selects relevant features during classifier construction by penalizing each feature’s use in the dual formulation of support vector machines (SVM). This approach called kernel-penalized SVM (KP-SVM) optimizes the shape of an anisotropic RBF Kernel eliminating features that have low relevance for the classifier. Additionally, KP-SVM employs an explicit stopping condition, avoiding the elimination of features that would negatively affect the classifier’s performance. We performed experiments on four real-world benchmark problems comparing our approach with well-known feature selection techniques. KP-SVM outperformed the alternative approaches and determined consistently fewer relevant features.     

Hierarchical classification and feature reduction for fast face detection with support vector machines

We present a two-step method to speed-up object detection systems in computer vision that use support vector machines as classifiers. In the first step we build a hierarchy of classifiers. On the bottom level, a simple and fast linear classifier analyzes the whole image and rejects large parts of the background. On the top level, a slower but more accurate classifier performs the final detection. We propose a new method for automatically building and training a hierarchy of classifiers. In the second step we apply feature reduction to the top level classifier by choosing relevant image features according to a measure derived from statistical learning theory. Experiments with a face detection system show that combining feature reduction with hierarchical classification leads to a speed-up by a factor of 335 with similar classification performance.     

基于GEP的最小二乘支持向量机模型参数选择

针对最小二乘支持向量机的多参数寻优问题,提出了一种基于基因表达式编程的最小二乘支持向量机参数优选方法.该算法将最小二乘支持向量机参数(C,σ)样本作为GEP的基因,按其变异算子随着进化代数和染色体所含基因数目动态变化的机制执行,其收敛速度和精确度大大提高.并与基于粒子群算法和遗传算法参数优选方法比较,通过标准测试函数验证了该算法的拟合误差最低.最后用其建立氧化铝生产蒸发过程参数预测模型,应用工业生产数据进行验证,实验结果表明该方法有效且获得了满意的效果.     

Feature selection using support vector machines and bootstrap methods for ventricular fibrillation detection

Early detection of ventricular fibrillation (VF) is crucial for the success of the defibrillation therapy in automatic devices. A high number of detectors have been proposed based on temporal, spectral, and time-frequency parameters extracted from the surface electrocardiogram (ECG), showing always a limited performance. The combination ECG parameters on different domain (time, frequency, and time-frequency) using machine learning algorithms has been used to improve detection efficiency. However, the potential utilization of a wide number of parameters benefiting machine learning schemes has raised the need of efficient feature selection (FS) procedures. In this study, we propose a novel FS algorithm based on support vector machines (SVM) classifiers and bootstrap resampling (BR) techniques. We define a backward FS procedure that relies on evaluating changes in SVM performance when removing features from the input space. This evaluation is achieved according to a nonparametric statistic based on BR. After simulation studies, we benchmark the performance of our FS algorithm in AHA and MIT-BIH ECG databases. Our results show that the proposed FS algorithm outperforms the recursive feature elimination method in synthetic examples, and that the VF detector performance improves with the reduced feature set.     

非线性特征提取和LSSVM在化工过程故障诊断中应用

提出利用非线性特征提取（核主成分分析（KPCA）和核独立成分分析）消除数据的不相关性,降低维数。核主成分分析利用核函数把输入数据映射到特征空间,进行线性主成分分析计算提取特征;核独立成分分析在KPCA白化空间进行线性独立成分分析（ICA）变换提取独立成分。提取的特征作为最小二乘支持向量机分类器的输入,构建融合非线性特征提取和最小二乘支持向量机的智能故障分类方法。研究了该方法应用到某石化企业润滑油生产过程的故障诊断中的有效性和可行性。     

Parameter determination of support vector machine and feature selection using simulated annealing approach

Support vector machine (SVM) is a novel pattern classification method that is valuable in many applications. Kernel parameter setting in the SVM training process, along with the feature selection, significantly affects classification accuracy. The objective of this study is to obtain the better parameter values while also finding a subset of features that does not degrade the SVM classification accuracy. This study develops a simulated annealing (SA) approach for parameter determination and feature selection in the SVM, termed SA-SVM.To measure the proposed SA-SVM approach, several datasets in UCI machine learning repository are adopted to calculate the classification accuracy rate. The proposed approach was compared with grid search which is a conventional method of performing parameter setting, and various other methods. Experimental results indicate that the classification accuracy rates of the proposed approach exceed those of grid search and other approaches. The SA-SVM is thus useful for parameter determination and feature selection in the SVM.     

ECG quality assessment based on a kernel support vector machine and genetic algorithm with a feature matrix

We propose a systematic ECG quality classification method based on a kernel support vector machine（KSVM） and genetic algorithm（GA） to determine whether ECGs collected via mobile phone are acceptable or not. This method includes mainly three modules, i.e., lead-fall detection, feature extraction, and intelligent classification. First, lead-fall detection is executed to make the initial classification. Then the power spectrum, baseline drifts, amplitude difference, and other time-domain features for ECGs are analyzed and quantified to form the feature matrix. Finally, the feature matrix is assessed using KSVM and GA to determine the ECG quality classification results. A Gaussian radial basis function（GRBF） is employed as the kernel function of KSVM and its performance is compared with that of the Mexican hat wavelet function（MHWF）. GA is used to determine the optimal parameters of the KSVM classifier and its performance is compared with that of the grid search（GS） method. The performance of the proposed method was tested on a database from PhysioNet/Computing in Cardiology Challenge 2011, which includes 1500 12-lead ECG recordings. True positive（TP）, false positive（FP）, and classification accuracy were used as the assessment indices. For training database set A（1000 recordings）, the optimal results were obtained using the combination of lead-fall, GA, and GRBF methods, and the corresponding results were： TP 92.89%, FP 5.68%, and classification accuracy 94.00%. For test database set B（500 recordings）, the optimal results were also obtained using the combination of lead-fall, GA, and GRBF methods, and the classification accuracy was 91.80%.     

基于SVM动态集成的高光谱遥感图像分类

在Bagging支持向量机(SVM)的基础上,将动态分类器集选择技术用于SVM的集成学习,研究了SVM动态集成在高光谱遥感图像分类中的应用。结合高光谱数据特性,通过随机选取特征子空间和反馈学习改进了Bagging SVM方法;通过引进加性复合距离改善了K近邻局部空间的计算方法;通过将错分的训练样本添加到验证集增强了验证集样本的代表性。实验结果表明,与单个优化的SVM和其他常见的SVM集成方法相比,改进后的SVM动态集成分类精度最高,能有效地提高高光谱遥感图像的分类精度。     

Evaluation of feature selection methods for text classification with small datasets using multiple criteria decision-making methods

The evaluation of feature selection methods for text classification with small sample datasets must consider classification performance, stability, and efficiency. It is, thus, a multiple criteria decision-making (MCDM) problem. Yet there has been few research in feature selection evaluation using MCDM methods which considering multiple criteria. Therefore, we use MCDM-based methods for evaluating feature selection methods for text classification with small sample datasets. An experimental study is designed to compare five MCDM methods to validate the proposed approach with 10 feature selection methods, nine evaluation measures for binary classification, seven evaluation measures for multi-class classification, and three classifiers with 10 small datasets. Based on the ranked results of the five MCDM methods, we make recommendations concerning feature selection methods. The results demonstrate the effectiveness of the used MCDM-based method in evaluating feature selection methods.     

基于一种新的特征提取法和支持向量机的膜蛋白分类研究

引入加权思想,以一种新的特征提取法,即加权自相关函数,表示蛋白质序列,与支持向量机组合,并采用“一对多”、“一对一”分类策略对膜蛋白进行分类研究,结果有明显改善。在采用支持向量机算法及“一对多”分类策略下,加权自相关函数特征提取法的每一类别分类精度、Matthews相关系数和总分类精度都要高于氨基酸组成成分特征提取法相应的分类结果, 其总分类精度和脂链锚锭蛋白的分类精度分别为87．98％、65．85％,比氨基酸组成成分特征提取法分别提高3．38、9．75个百分点;“一对一”策略的总分类精度可达到94．88％,比“一对多”策略提高6．9个百分点;支持向量机机器学习算法的分类能力优于贝叶斯协方差统计算法,其总分类精度比贝叶斯协方差算法最大可提高15．6个百分点。     

Towards an accurate sleep apnea detection based on ECG signal: The quintessential of a wise feature selection

A wise feature selection from minute-to-minute Electrocardiogram (ECG) signal is a challenging task for many reasons, but mostly because of the promise of the accurate detection of clinical disorders, such as the sleep apnea. In this study, the ECG signal was modeled in order to obtain the Heart Rate Variability (HRV) and the ECG-Derived Respiration (EDR). Selected features techniques were used for benchmark with different classifiers such as Artificial Neural Networks (ANN) and Support Vector Machine(SVM), among others. The results evidence that the best accuracy was 82.12%, with a sensitivity and specificity of 88.41% and 72.29%, respectively. In addition, experiments revealed that a wise feature selection may improve the system accuracy. Therefore, the proposed model revealed to be reliable and simpler alternative to classical solutions for the sleep apnea detection, for example the ones based on the Polysomnography.     

Fractional order fuzzy-PID control of a combined cycle power plant using Particle Swarm Optimization algorithm with an improved dynamic parameters selection

The effectiveness of the Particle Swarm Optimization (PSO) algorithm in solving any optimization problem is highly dependent on the right selection of tuning parameters. A better control parameter improves the flexibility and robustness of the algorithm. In this paper, a new PSO algorithm based on dynamic control parameters selection is presented in order to further enhance the algorithm's rate of convergence and the minimization of the fitness function. The powerful Dynamic PSO (DPSO) uses a new mechanism to dynamically select the best performing combinations of acceleration coefficients, inertia weight, and population size. A fractional order fuzzy-PID (fuzzy-FOPID) controller based on the DPSO algorithm is proposed to perform the optimization task of the controller gains and improve the performance of a single-shaft Combined Cycle Power Plant (CCPP). The proposed controller is used in speed control loop to improve the response during frequency drop or change in loading. The performance of the fuzzy-FOPID based DPSO is compared with those of the conventional PSO, Comprehensive Learning PSO (CLPSO), Heterogeneous CLPSO (HCLPSO), Genetic Algorithm (GA), Differential Evolution (DE), and Artificial Bee Colony (ABC) algorithm. The simulation results show the effectiveness and performance of the proposed method for frequency drop or change in loading.     

Faults diagnosis of a centrifugal pump using multilayer perceptron genetic algorithm back propagation and support vector machine with discrete wavelet transform‐based feature extraction

This paper presents a comparative study of two artificial intelligent systems, namely; Multilayer Perceptron (MLP) and support vector machine (SVM), to classify six fault conditions and the normal (nonfaulty) condition of a centrifugal pump. A hybrid training method for MLP is proposed for this work based on the combination of Back Propagation (BP) and Genetic Algorithm (GA). The two training algorithms are tested and compared separately as well. Features are extracted using Discrete Wavelet Transform (DWT), both approximations, details, and two mother wavelets were used to investigate their effectiveness on feature extraction. GA is also used to optimize the number of hidden layers and neurons of MLP. In this study, the feature extraction, GA‐based hidden layers, neurons selection, training algorithm, and classification performance, based on the strengths and weaknesses of each method, are discussed. From the results obtained, it is observed that the DWT with both MLP‐BP and SVM produces better classification rates and performances.