支持向量机多分类算法的研究

支持向量机（support vector machine,SVM）是在统计学习理论基础上发展起来的一种新的数据挖掘方法,并已广泛应用于模式识别与回归分析等领域。本文重点阐述了一些典型的支持向量机多分类算法及支持向量机多标注算法。最后指出了进一步研究和亟待解决的一些问题。     

多分类孪生支持向量机研究进展

孪生支持向量机因其简单的模型、快速的训练速度和优秀的性能而受到广泛关注.该算法最初是为解决二分类问题而提出的,不能直接用于解决现实生活中普遍存在的多分类问题.近来,学者们致力于将二分类孪生支持向量机扩展为多分类方法并提出了多种多分类孪生支持向量机.多分类孪生支持向量机的研究已经取得了一定的进展.本文主要工作是回顾多分类孪生支持向量机的发展,对多分类孪生支持向量机进行合理归类,分析各个类型的多分类孪生支持向量机的理论和几何意义.本文以多分类孪生支持向量机的子分类器组织结构为依据,将多分类孪生支持向量机分为：基于“一对多”策略的多分类孪生支持向量机、基于“一对一”策略的多分类孪生支持向量机、基于“一对一对余”策略的多分类孪生支持向量机、基于二叉树结构的多分类孪生支持向量机和基于“多对一”策略的多分类孪生支持向量机.基于有向无环图的多分类孪生支持向量机训练过程与基于“一对一”策略的多分类孪生支持向量机类似,但是其决策方式有其特殊的优缺点,因此本文将其也独立为一类.本文分析和总结了这六种类型的多分类孪生支持向量机的算法思想、理论基础.此外,还通过实验对比了分类性能.本文工作为各种多分类孪生支持向量机之间建立了联系比较,使得初学者能够快速理解不同多分类孪生支持向量机之间的本质区别,也对实际应用中选取合适的多分类孪生支持向量机起到一定的指导作用.     

支持向量机在多类分类问题中的推广

支持向量机(SVMs)最初是用以解决两类分类问题,不能直接用于多类分类,如何有效地将其推广到多类分类问题是一个正在研究的问题。该文总结了现有主要的支持向量机多类分类算法,系统地比较了各算法的训练速度、分类速度和推广能力,并分析它们的不足和有待解决的问题。     

一种新的多类支持向量机算法

提出了一种新的多类支持向量机算法OC-K-SVM.对k类分类问题,该方法构造了k个分类器,每一个分类器只对一类样本进行训练.使用Benchmark的数据集进行了初步的实验,实验结果验证了算法的有效性.     

多类支持向量机推广性能分析

为了分析多类支持向量机(Multi-category support vector machines,M-SVMs)的推广性能,对常用的M-SVMs算法加以概述,推导、总结了理论推广误差公式.对于给定的样本集,可以设计合理的编码来提高ECOCSVMs的推广性能,通过构造合理的层次结构来提高H-SVMs推广性能,其余M-SVMs算法的推广性能均取决于样本空间.研究结果为有效使用M-SVMs提供了依据,为改进M-SVMs指明了方向.     

支持向量机多类分类算法研究

提出一种新的基于二叉树结构的支持向量(SVM)多类分类算法.该算法解决了现有主要算法所存在的不可分区域问题.为了获得较高的推广能力,必须让样本分布广的类处于二叉树的上层节点,才能获得更大的划分空间.所以,该算法采用最小超立方体和最小超球体类包含作为二叉树的生成算法.实验结果表明,该算法具有一定的优越性.     

支持向量机多类分类方法

支持向量机本身是一个两类问题的判别方法,不能直接应用于多类问题。当前针对多类问题的支持向量机分类方法主要有5种：一类对余类法（OVR）,一对一法（OVO）,二叉树法（BT）,纠错输出编码法和有向非循环图法。本文对这些方法进行了简单的介绍,通过对其原理和实现方法的分析,从速度和精度两方面对这些方法的优缺点进行了归纳和总结,给出了比较意见,并通过实验进行了验证,最后提出了一些改进建议。     

多类支持向量机在文本分类中的应用

传统的支持向量机（SVM）是两类分类问题,如何有效地将其推广到多类分类问题仍是一项有待研究的课题。本文在对现有主要的四种多类支持向量机分类算法讨论的基础上,结合文本分类的特点,详细介绍了决策树支持向量机和几种改进多类支持向量机方法在文本分类中的应用。     

基于粒度偏移因子的支持向量机学习方法

在实际应用中,数据集样本规模、分布密度的不平衡性可能会使传统支持向量机(support vector machine, SVM)得到的分类超平面不是最优.在对传统支持向量机最优分类面分析的基础上,结合粒度计算(granular computing, GrC)理论,针对数据规模和分布密度不平衡的数据集,提出一种基于粒度偏移因子的粒度支持向量机(granular SVM, GSVM)学习方法,称为S_GSVM方法.该方法将原始样本用Mercer核映射到高维空间,然后在高维空间中对数据进行有效的粒划分,通过对不同的粒计算不同的超平面偏移因子,重新构造支持向量机的凸二次优化问题,以得到一个泛化能力更好的分类超平面.S_GSVM方法充分考虑了数据复杂分布对于泛化能力的影响,对基于最大间隔的分类面进行改进.实验结果表明,S_GSVM方法在非平衡数据集上能得到较好的泛化性能.     

多类支持向量机文本分类方法

文本分类是数据挖掘的基础和核心,支持向量机（SVM）是解决文本分类问题的最好算法之一。传统的支持向量机是两类分类问题,如何有效地将其推广到多类分类问题仍是一项有待研究的课题。介绍了支持向量机的基本原理,对现有主要的多类支持向量机文本分类算法进行了讨论和比较。提出了多类支持向量机文本分类中存在的问题和今后的发展。     

Automatic model selection for the optimization of SVM kernels

This approach aims to optimize the kernel parameters and to efficiently reduce the number of support vectors, so that the generalization error can be reduced drastically. The proposed methodology suggests the use of a new model selection criterion based on the estimation of the probability of error of the SVM classifier. For comparison, we considered two more model selection criteria: GACV (‘Generalized Approximate Cross-Validation’) and VC (‘Vapnik-Chernovenkis’) dimension. These criteria are algebraic estimates of upper bounds of the expected error. For the former, we also propose a new minimization scheme. The experiments conducted on a bi-class problem show that we can adequately choose the SVM hyper-parameters using the empirical error criterion. Moreover, it turns out that the criterion produces a less complex model with fewer support vectors. For multi-class data, the optimization strategy is adapted to the one-against-one data partitioning. The approach is then evaluated on images of handwritten digits from the USPS database.     

Model selection for primal SVM

This paper introduces two types of nonsmooth optimization methods for selecting model hyperparameters in primal SVM models based on cross-validation. Unlike common grid search approaches for model selection, these approaches are scalable both in the number of hyperparameters and number of data points. Taking inspiration from linear-time primal SVM algorithms, scalability in model selection is achieved by directly working with the primal variables without introducing any dual variables. The proposed implicit primal gradient descent (ImpGrad) method can utilize existing SVM solvers. Unlike prior methods for gradient descent in hyperparameters space, all work is done in the primal space so no inversion of the kernel matrix is required. The proposed explicit penalized bilevel programming (PBP) approach optimizes both the hyperparameters and parameters simultaneously. It solves the original cross-validation problem by solving a series of least squares regression problems with simple constraints in both the hyperparameter and parameter space. Computational results on least squares support vector regression problems with multiple hyperparameters establish that both the implicit and explicit methods perform quite well in terms of generalization and computational time. These methods are directly applicable to other learning tasks with differentiable loss functions and regularization functions. Both the implicit and explicit algorithms investigated represent powerful new approaches to solving large bilevel programs involving nonsmooth loss functions.     

An improved code selection algorithm for fault prediction

The goal of this study is to present an improved code selection algorithm (BCSA) for fault prediction. The contributions mainly contain three parts. The first part is on the extension of the horizontal input in the code selection algorithm (CSA). We propose that the horizontal input is also the prediction for the next coming event, not only for recalling. Thus, BCSA is able to recall and predict alternately. The second part is on the extension of the generic minicolumnar function. We propose that the function of a minicolumn is to be a k-winner-take-all competitive module (CM) and all active cells (the overall input is 1) should be chosen as winners within a CM. The third part is on the improvement of the competition mechanism. In BCSA, the winners are directly chosen with only one round competition. Thus, computing the input’s similarity G is unnecessary. BCSA is applied to analyze the disaster of the space shuttle Challenger which is a well-known example of fault prediction. Compared to other methods, the result of BCSA is specific, robust and independent of the parameters.     

Multi-objective uniform design as a SVM model selection tool for face recognition

The primary difficulty of support vector machine (SVM) model selection is heavy computational cost, thus it is difficult for current model selection methods to be applied in face recognition. Model selection via uniform design can effectively alleviate the computational cost, but its drawback is that it adopts a single objective criterion which can not always guarantee the generalization capacity. The sensitivity and specificity as multi-objective criteria have been proved of better performance and can provide a means for obtaining more realistic models. This paper first proposes a multi-objective uniform design (MOUD) search method as a SVM model selection tool, and then applies this optimized SVM classifier to face recognition. Because of replacing single objective criterion with multi-objective criteria and adopting uniform design to seek experimental points that uniformly scatter on whole experimental domain, MOUD can reduce the computational cost and improve the classification ability simultaneously. The experiments are executed on UCI benchmark, and on Yale and CAS-PEAL-R1 face databases. The experimental results show that the proposed method outperforms other model search methods significantly, especially for face recognition.     

Lagrangian relaxation for SVM feature selection

We discuss a Lagrangian-relaxation-based heuristics for dealing with feature selection in the Support Vector Machine (SVM) framework for binary classification. In particular we embed into our objective function a weighted combination of the L₁ and L₀ norm of the normal to the separating hyperplane. We come out with a Mixed Binary Linear Programming problem which is suitable for a Lagrangian relaxation approach.Based on a property of the optimal multiplier setting, we apply a consolidated nonsmooth optimization ascent algorithm to solve the resulting Lagrangian dual. In the proposed approach we get, at every ascent step, both a lower bound on the optimal solution as well as a feasible solution at low computational cost.We present the results of our numerical experiments on some benchmark datasets.     

Modified binary PSO for feature selection using SVM applied to mortality prediction of septic patients

This paper proposes a modified binary particle swarm optimization (MBPSO) method for feature selection with the simultaneous optimization of SVM kernel parameter setting, applied to mortality prediction in septic patients. An enhanced version of binary particle swarm optimization, designed to cope with premature convergence of the BPSO algorithm is proposed. MBPSO control the swarm variability using the velocity and the similarity between best swarm solutions. This paper uses support vector machines in a wrapper approach, where the kernel parameters are optimized at the same time. The approach is applied to predict the outcome (survived or deceased) of patients with septic shock. Further, MBPSO is tested in several benchmark datasets and is compared with other PSO based algorithms and genetic algorithms (GA). The experimental results showed that the proposed approach can correctly select the discriminating input features and also achieve high classification accuracy, specially when compared to other PSO based algorithms. When compared to GA, MBPSO is similar in terms of accuracy, but the subset solutions have less selected features.     

克隆代码有害性预测中的特征选择模型

为解决克隆代码有害性预测过程中特征无关与特征冗余的问题,提出一种基于相关程度和影响程度的克隆代码有害性特征选择组合模型。首先,利用信息增益率对特征数据进行相关性的初步排序;然后,保留相关性排名较高的特征并去除其他无关特征,减小特征的搜索空间;接着,采用基于朴素贝叶斯等六种分类器分别与封装型序列浮动前向选择算法结合来确定最优特征子集。最后对不同的特征选择方法进行对比分析,将各种方法在不同选择准则上的优势加以利用,对特征数据进行分析、筛选和优化。实验结果表明,与未进行特征选择之前对比发现有害性预测准确率提高15.2~34个百分点以上;与其他特征选择方法比较,该方法在F1测度上提高1.1~10.1个百分点,在AUC指标上提升达到0.7~22.1个百分点,能极大地提高有害性预测模型的准确度。     

基于PAC-Bayes边界理论的SVM模型选择方法

PAC-Bayes边界理论融合了贝叶斯定理和随机分类器的结构风险最小化原理,它作为一个理论框架,能有效评价机器学习算法的泛化性能。针对支持向量机（SVM）模型选择问题,通过分析PAC-Bayes边界理论框架及其在SVM上的应用,将PAC-Bayes边界理论与基于交叉验证的网格搜索法相结合,提出一种基于PAC-Bayes边界的SVM模型选择方法（PBB-GS）,实现快速优选SVM的惩罚系数和核函数参数。UCI数据集的实验结果表明该方法优选出的参数能使SVM具有较高的泛化性能,并具有简便快速、参数选择准确的优点,能有效改善SVM模型选择问题。     

An improvement on floating search algorithms for feature subset selection

A new improved forward floating selection (IFFS) algorithm for selecting a subset of features is presented. Our proposed algorithm improves the state-of-the-art sequential forward floating selection algorithm. The improvement is to add an additional search step called “replacing the weak feature” to check whether removing any feature in the currently selected feature subset and adding a new one at each sequential step can improve the current feature subset. Our method provides the optimal or quasi-optimal (close to optimal) solutions for many selected subsets and requires significantly less computational load than optimal feature selection algorithms. Our experimental results for four different databases demonstrate that our algorithm consistently selects better subsets than other suboptimal feature selection algorithms do, especially when the original number of features of the database is large.     

SVM与神经网络模型在股票预测中的应用研究

介绍了SVM、BP神经网络和小波神经网络模型在股票预测中的应用研究。通过输入历史股票价格走势数据进行模型训练,并分别进行三个模型预测输出,最后通过均方误差、走势方向准确率和总盈利率三个指标分析比较三个模型,从而了解模型在股票预测领域的应用效果,为后续研究做参考。