非平衡二叉树多类支持向量机分类方法

提出一种新的基于非平衡二叉树的支持向量机多类别分类方法。该方法通过分析已知类别样本的先验分布知识,构造一个二叉决策树,使容易区分的类别从根节点开始逐层分割出来,以获得较高的推广能力。该方法解决了传统分类算法中所存在的不可分区域问题,在训练时只需构造N-1个SVM分类器,而测试时的判决次数小于N。将该方法应用于人脸识别实验。测试结果表明,与传统分类算法相比,该方法的平均分类时间是最少的。     

多类支持向量机分类算法—DDAG①

随着支持向量机的发展,由最初的两类分类问题逐渐推广到多类分类问题,且其思想、算法多种多样,各有千秋。主要研究以当前比较流行的以多个二类分类器组合实现多类分类器的算法之一：DDAG。提出此算法在多类支持向量机应用分类中存在的优点和不足,并针对其不足,提出一种改进的算法思想。     

基于后验概率SVM决策树多类的分类算法

后验概率支持向量机方法对孤立点和噪声具有鲁棒性,并且可以减少支持向量的数量,从而降低计算复杂度。因此,针对最近提出的快速分类算法c-BTS,引入样本的后验概率,提出了一种基于后验概率的SVM决策树算法P2BTS。实验结果证明,基于后验概率的支持向量机决策树P2BTS比c-BTS的分类精度更高,且所需的二类分类器个数减少,在一定程度上降低了P2BTS分类决策的时间和比较的次数,提高了分类效率。     

基于球结构的完全二叉树SVM多类分类算法*

针对一般的SVM方法不能有效地处理不平衡样本数据及现有的偏二叉树结构SVM分类器速度慢的这两个问题,提出了一种基于球结构的完全二叉树SVM多分类算法。该算法利用球结构的SVM考虑了每个类的分布情况,能有效地处理不平衡样本数据;构建完全二叉树结构,使得同层节点所代表的SVM分类器可以并行工作,能提高其训练和分类速度,分类速度相当于折半查找。实例验证两者结合后的算法可实现准确且高效的多类分类。     

基于超球和ASSRFOA的多生支持向量机

支持向量机(support vector machine, SVM)是一种基于结构风险最小化的机器学习方法, 能够有效解决分类问题. 但随着研究问题的复杂化, 现实的分类问题往往是多分类问题, 而SVM仅能用于处理二分类任务. 针对这个问题, 一对多策略的多生支持向量机(multiple birth support vector machine, MBSVM)能够以较低的复杂度实现多分类, 但缺点在于分类精度较低. 本文对MBSVM进行改进, 提出了一种新的SVM多分类算法: 基于超球(hypersphere)和自适应缩小步长果蝇优化算法(fruit fly optimization algorithm with adaptive step size reduction, ASSRFOA)的MBSVM, 简称HA-MBSVM. 通过拟合超球得到的信息, 先进行类别划分再构建分类器, 并引入约束距离调节因子来适当提高分类器的差异性, 同时采用ASSRFOA求解二次规划问题, HA-MBSVM可以更好地解决多分类问题. 我们采用6个数据集评估HA-MBSVM的性能, 实验结果表明HA-MBSVM的整体性能优于各对比算法.     

A cellular automata-based learning method for classification

Over the last few decades, classification applied to numerous applications in science, engineering, business and industries have rapidly been increased, especially for big data. However, classifiers dealing with complicated high dimension problems with non-conforming patterns with high accuracy are rare, especially for bit-level features. It is a challenging research problem. This paper proposed a novel efficient classifier based on cellular automata model, called Cellular Automata-based Classifier (CAC). CAC possesses the promising capability to deal with non-conforming patterns in the bit-level features. It was developed on a new kind of the proposed elementary cellular automata, called Decision Support Elementary Cellular Automata (DS-ECA). The classification capability of DS-ECA is promising since it can describe very complicated decision rule in high dimension problems with less complexity. CAC comprises double rule vectors and a decision function, the structure of which has two layers; the first layer is employed to evolve an input pattern into feature space and the other interprets the patterns in feature space as binary answer through the decision function. It has a time complexity of learning at O(n²), while the classification for one instance is O(1), where n is a number of bit patterns. For classification performance, 12 datasets consisting of binary and non-binary features are empirically implemented in comparison with Support Vector Machines (SVM) using k-fold cross validation. In this respect, CAC outperforms SVM with the best kernel for binary features, and provides the promising results equivalent to SVM on average for non-binary features.     

A comparative study for content-based dynamic spam classification using four machine learning algorithms

The growth of email users has resulted in the dramatic increasing of the spam emails during the past few years. In this paper, four machine learning algorithms, which are Naïve Bayesian (NB), neural network (NN), support vector machine (SVM) and relevance vector machine (RVM), are proposed for spam classification. An empirical evaluation for them on the benchmark spam filtering corpora is presented. The experiments are performed based on different training set size and extracted feature size. Experimental results show that NN classifier is unsuitable for using alone as a spam rejection tool. Generally, the performances of SVM and RVM classifiers are obviously superior to NB classifier. Compared with SVM, RVM is shown to provide the similar classification result with less relevance vectors and much faster testing time. Despite the slower learning procedure, RVM is more suitable than SVM for spam classification in terms of the applications that require low complexity.     

基于后验概率的SVM决策树多类分类算法

后验概率支持向量机方法对孤立点和噪声具有鲁棒性,并且可以减少支持向量的数量,从而降低计算复杂度。因此,针对最近提出的快速分类算法c-BTS,引入样本的后验概率,提出了一种基于后验概率的SVM决策树算法P2BTS。实验结果证明,基于后验概率的支持向量机决策树P2BTS比c-BTS的分类精度更高,且所需的二类分类器个数减少,在一定程度上降低了P2BTS分类决策的时间和比较的次数,提高了分类效率。     

迭代再权q范数正则化LS SVM分类算法

提出一种迭代再权q范数正则化最小二乘支持向量机(LS SVM)分类算法。该算法通过交叉校验过程选择正则化范数的阶次q (0相似文献   

A comparison of methods for multiclass support vector machines

Support vector machines (SVMs) were originally designed for binary classification. How to effectively extend it for multiclass classification is still an ongoing research issue. Several methods have been proposed where typically we construct a multiclass classifier by combining several binary classifiers. Some authors also proposed methods that consider all classes at once. As it is computationally more expensive to solve multiclass problems, comparisons of these methods using large-scale problems have not been seriously conducted. Especially for methods solving multiclass SVM in one step, a much larger optimization problem is required so up to now experiments are limited to small data sets. In this paper we give decomposition implementations for two such "all-together" methods. We then compare their performance with three methods based on binary classifications: "one-against-all," "one-against-one," and directed acyclic graph SVM (DAGSVM). Our experiments indicate that the "one-against-one" and DAG methods are more suitable for practical use than the other methods. Results also show that for large problems methods by considering all data at once in general need fewer support vectors.     

基于对支持向量机的多类分类算法在入侵检测中的应用

针对基于传统支持向量机(SVM)的多类分类算法在处理大规模数据时训练速度上存在的弱势,提出了一种基于对支持向量机(TWSVM)的多类分类算法。该算法结合二叉树SVM多类分类思想,通过在二叉树节点处构造基于TWSVM的分类器来达到分类目的。为减少二叉树SVM的误差累积,算法分类前首先通过聚类算法得到各类的聚类中心,通过比较各聚类中心之间的距离来衡量样本的差异以决定二叉树节点处类别的分离顺序,最后将算法用于网络入侵检测。实验结果表明,该算法不仅保持了较高的检测精度,在训练速度上还表现了一定优势,尤其在处理稍大规模数据时,这种优势更为明显,是传统二叉树SVM多类分类算法训练速度的近两倍,为入侵检测领域大规模数据处理提供了有效参考价值。     

Large data sets classification using convex–concave hull and support vector machine

Normal support vector machine (SVM) is not suitable for classification of large data sets because of high training complexity. Convex hull can simplify the SVM training. However, the classification accuracy becomes lower when there exist inseparable points. This paper introduces a novel method for SVM classification, called convex–concave hull SVM (CCH-SVM). After grid processing, the convex hull is used to find extreme points. Then, we use Jarvis march method to determine the concave (non-convex) hull for the inseparable points. Finally, the vertices of the convex–concave hull are applied for SVM training. The proposed CCH-SVM classifier has distinctive advantages on dealing with large data sets. We apply the proposed method on several benchmark problems. Experimental results demonstrate that our approach has good classification accuracy while the training is significantly faster than other SVM classifiers. Compared with the other convex hull SVM methods, the classification accuracy is higher.     

A Novel and Principled Multiclass Support Vector Machine

Support vector machines (SVMs) have been demonstrated very efficient for binary classification problems; however, computationally efficient and effective multiclass SVMs are still missing. Most existing multiclass SVM classifiers are constructed either by combining multiple binary SVM classifiers, which often perform moderately for some problems, or by converting multiclass problems into one single optimization problem, which is unfortunately computationally expensive. To address these issues, a novel and principled multiclass SVM based on geometric properties of hyperspheres, termed SVMGH, is proposed in this paper. Different from existing SVM‐based methods that seek a cutting hyperplane between two classes, SVMGH draws the discriminative information of each class by constructing a minimum hypersphere containing all class members, and then defines a label function based on the geometric properties of the minimum hyperspheres. We prove theoretically the geometric properties of the minimum hyperspheres to guarantee the validation of SVMGH. The computational efficiency is enhanced by a data reduction strategy as well as a fast training method. Experimental results demonstrate that the proposed SVMGH shows better performance and higher computational efficiency than the state of the art on multiclassification problems while maintaining comparable performance and efficiency on binary classification problems.     

Fast Multiclass SVM Classification Using Decision Tree Based One-Against-All Method

We present an improved version of One-Against-All (OAA) method for multiclass SVM classification based on a decision tree approach. The proposed decision tree based OAA (DT-OAA) is aimed at increasing the classification speed of OAA by using posterior probability estimates of binary SVM outputs. DT-OAA decreases the average number of binary SVM tests required in testing phase to a greater extent when compared to OAA and other multiclass SVM methods. For a balanced multiclass dataset with K classes, under best situation, DT-OAA requires only (K + 1)/2 binary tests on an average as opposed to K binary tests in OAA; however, on imbalanced multiclass datasets we observed DT-OAA to be much faster with proper selection of order in which the binary SVMs are arranged in the decision tree. Computational comparisons on publicly available datasets indicate that the proposed method can achieve almost the same classification accuracy as that of OAA, but is much faster in decision making.     

基于字符匹配度的SVM多分类器设计简

本文介绍了常用的SVM多分类器的构造思路,并且分析比较了各种思路的优缺点。针对二叉树决策法中强制分类存在的误判情况,提出了基于字符匹配度的SVM多分类器设计方案。通过实验对比各种多分类器的分类器数量、平均分类次数、训练和判断时间、识别正确率,证明了该构造方案的优越性。     

基于概率投票策略的多类支持向量机及应用

传统的支持向量机是基于两类问题提出的,如何将其有效地推广至多类分类仍是一个研究的热点问题。在分析比较现有支持向量机多类分类OVO方法存在的问题及缺点的基础上,该文提出一种新的基于概率投票策略的多类分类方法。在该策略中,充分考虑了OVO方法中各个两类支持向量机分类器的差异,并将该差异反映到投票分值上。所提多类支持向量机方法不仅具有较好的分类性能,而且有效解决了传统投票策略中存在的拒分区域问题。将基于概率投票的多分类支持向量机作为关键技术应用于实际齿轮箱故障诊断,并与传统投票策略的结果进行对比,表明所提方法的上述优点。     

一种新的多类支持向量机决策方法及其应用

讨论和比较了现有的几种多类SVM方法．在此基础上,提出了一种组合多个两类分类器结果的多类SVM决策方法．在该方法中,定义了新的决策函数,其值是在传统投票决策值的基础上乘以不同分类器的权重．新的多类SVM在一定程度上解决了传统投票决策方法的不可分区域问题,因此具有更好的分类性能．最后,将新方法作为关键技术应用于故障诊断实例,实际诊断结果证明了所提多类SVM决策方法的优越性．     

Benchmarking Least Squares Support Vector Machine Classifiers

In Support Vector Machines (SVMs), the solution of the classification problem is characterized by a (convex) quadratic programming (QP) problem. In a modified version of SVMs, called Least Squares SVM classifiers (LS-SVMs), a least squares cost function is proposed so as to obtain a linear set of equations in the dual space. While the SVM classifier has a large margin interpretation, the LS-SVM formulation is related in this paper to a ridge regression approach for classification with binary targets and to Fisher's linear discriminant analysis in the feature space. Multiclass categorization problems are represented by a set of binary classifiers using different output coding schemes. While regularization is used to control the effective number of parameters of the LS-SVM classifier, the sparseness property of SVMs is lost due to the choice of the 2-norm. Sparseness can be imposed in a second stage by gradually pruning the support value spectrum and optimizing the hyperparameters during the sparse approximation procedure. In this paper, twenty public domain benchmark datasets are used to evaluate the test set performance of LS-SVM classifiers with linear, polynomial and radial basis function (RBF) kernels. Both the SVM and LS-SVM classifier with RBF kernel in combination with standard cross-validation procedures for hyperparameter selection achieve comparable test set performances. These SVM and LS-SVM performances are consistently very good when compared to a variety of methods described in the literature including decision tree based algorithms, statistical algorithms and instance based learning methods. We show on ten UCI datasets that the LS-SVM sparse approximation procedure can be successfully applied.     

基于球向量机的中文文本分类

SVM在文本分类中的应用是近年来文本分类领域重要的进展之一。许多实验表明,SVM在文本分类中比其他的机器学习算法表现出更高的分类精度,但在大规模数据上的收敛速度较慢,成为SVM在实际应用中的一大缺点。球向量机是一种比SVM更快的机器学习方法。本文将BVM应用于文本分类。实验表明,BVM在文本分类中的应用具有与SVM相当的精 度,而且比SVM有更少的训练时间。     

Towards a comprehensive evaluation of V-I-S sub-pixel fractions and land surface temperature for urban land-use classification in the USA

Remote-sensing image classification based on the vegetation–impervious surface–soil (V-I-S) model and land-surface temperature (LST) has proved to be more efficient in characterizing the urban landscape than conventional spectral-based classification. However, current literature emphasizes discussion of the classifier's accuracy improvement achieved by the input of V-I-S fractions and LST over conventional spectral-based classification while ignoring the stability evaluation. Hence, this study proposes an evaluation framework for exploring the superiority of the input features and the stability of classifiers by integrating statistical randomization techniques and a kappa-error diagram. The evaluation framework was applied to case studies for demonstrating the superiority of V-I-S fractions and LST in the context of urban land-use classification with five different types of classifiers, including the maximum likelihood classifier (MLC), the tree classifier, the Bagging classifier, the random forest (RF) and the support vector machine (SVM). It followed that the use of V-I-S fractions and LST (1) could alleviate the ‘salt and pepper’ effect; (2) is preferred by tree and tree-based ensembles for branch splitting; (3) could produce classification trees with less complexity; (4) could benefit the stability of classifiers in addition to the accuracy improvement; and (5) could allow histograms following nearly normal distribution in its feature space, which boosts the performance of MLC. It is shown that MLC becomes comparable with modern classifiers when trained with V-I-S fractions and LST combination. Because of its adequacy and simplicity, MLC is recommended for urban land-use classification when V-I-S fractions and LST are used as the only input features. However, replacing them with, or including, the band reflectance might degrade MLC. A direct use of spectral band reflectance is not recommended for any of the classification approaches being considered in this study, except for SVM, which is the most robust classifier as it has a consistently high performance for all the input feature combinations. We recommend using tree-based ensemble classifiers or SVM when V-I-S fractions and LST as well as the band reflectance are all used in the classification. The proposed evaluation framework can also be applied to the assessment of input features and classifiers in other remote-sensing classification endeavours.