Functional iterative approaches for solving support vector classification problems based on generalized Huber loss

Classical support vector machine (SVM) and its twin variant twin support vector machine (TWSVM) utilize the Hinge loss that shows linear behaviour, whereas the least squares version of SVM (LSSVM) and twin least squares support vector machine (LSTSVM) uses L₂-norm of error which shows quadratic growth. The robust Huber loss function is considered as the generalization of Hinge loss and L₂-norm loss that behaves like the quadratic L₂-norm loss for closer error points and the linear Hinge loss after a specified distance. Three functional iterative approaches based on generalized Huber loss function are proposed in this paper to solve support vector classification problems of which one is based on SVM, i.e. generalized Huber support vector machine and the other two are in the spirit of TWSVM, namely generalized Huber twin support vector machine and regularization on generalized Huber twin support vector machine. The proposed approaches iteratively find the solutions and eliminate the requirements to solve any quadratic programming problem (QPP) as for SVM and TWSVM. The main advantages of the proposed approach are: firstly, utilize the robust Huber loss function for better generalization and for lesser sensitivity towards noise and outliers as compared to quadratic loss; secondly, it uses functional iterative scheme to find the solution that eliminates the need to solving QPP and also makes the proposed approaches faster. The efficacy of the proposed approach is established by performing numerical experiments on several real-world datasets and comparing the result with related methods, viz. SVM, TWSVM, LSSVM and LSTSVM. The classification results are convincing.

     

一种模糊加权的孪生支持向量机算法

虽然孪生支持向量机（Twin Support Vector Machine,TSVM）的处理速度优于传统的支持向量机,但其并没有考虑输入样本点对最优分类超平面所产生的不同影响。通过为每个训练样本赋予不同的样本重要性,以及减少样本点对非平行超平面的影响,提出了模糊加权孪生支持向量机（Fuzzy TSVM,FTSVM）。在UCI标准数据集上,对FTSVM进行了实验研究并与TSVM、FSVM和SVM方法进行了比较,实验结果表明FTSVM方法是有效的。     

基于混合模糊隶属度的模糊双支持向量机研究

双支持向量机是近年提出的一种新的支持向量机.在处理模式分类问题时,双支持向量机速度远远超过传统支持向量机,而且显示出较好的推广能力.但双支持向量机没有考虑不同输入样本点可能会对分类超平面的形成产生不同影响,在某些实际问题中具有局限性.为了克服这个缺点,提出了一种基于混合模糊隶属度的模糊双支持向量机.该算法设计了一种结合距离和紧密度的模糊隶属度函数,给不同的训练样本赋予不同的模糊隶属度,构建两个最优非平行超平面,最终实现二值分类.实验证明,该模糊双支持向量机的分类性能优于传统的双支持向量机.     

基于混合隶属度的模糊简约双支持向量机研究

双支持向量机是一种新的非平行二分类算法,其处理速度比传统支持向量机快很多,但是双支持向量机在训练之前要进行大量的复杂逆矩阵计算;在非线性情况下,它不能像传统支持向量机那样把核技巧直接运用到对偶优化问题中;并且双支持向量机没有考虑不同输入样本点会对最优分类超平面产生不同的影响。针对这些情况,提出了一种模糊简约双支持向量机。该模糊简约双支持向量机通过对二次规划函数和拉格朗日函数的改进,省略大量的逆矩阵计算,同时核技巧能直接运用到非线性分类情况下;对于混合模糊隶属度函数,不仅每个样本点到类中心的距离影响着该混合模糊隶属度,而且该样本点的邻域密度同样影响着该混合模糊隶属度。实验结果表明,与支持向量机、标准双支持向量机、双边界支持向量机、模糊双支持向量机相比,具有该混合模糊隶属度函数的简约双支持向量机不仅分类时间短,计算简单,而且分类精度高。     

面向不均衡分类的隶属度加权模糊支持向量机

针对不均衡分类问题,提出了一种基于隶属度加权的模糊支持向量机模型。使用传统支持向量机对样本进行训练,并通过样本点与所得分类超平面之间的距离构造模糊隶属度,这不仅能够消除噪点和野值点的影响,而且可以在一定程度上约减样本;利用正负类的平均隶属度和样本数量求得平衡调节因子,消除数据不平衡时造成的分类超平面的偏移现象;通过实验结果验证了该算法的可行性和有效性。实验结果表明,该算法能有效提高分类精度,特别是对不平衡数据效果更加明显,在训练速度和分类性能上比传统支持向量机和模糊支持向量机有进一步的提升。     

KNN-based least squares twin support vector machine for pattern classification

The least squares twin support vector machine (LSTSVM) generates two non-parallel hyperplanes by directly solving a pair of linear equations as opposed to solving two quadratic programming problems (QPPs) in the conventional twin support vector machine (TSVM), which makes learning speed of LSTSVM faster than that of the TSVM. However, LSTSVM fails to discover underlying similarity information within samples which may be important for classification performance. To address the above problem, we apply the similarity information of samples into LSTSVM to build a novel non-parallel plane classifier, called K-nearest neighbor based least squares twin support vector machine (KNN-LSTSVM). The proposed method not only retains the superior advantage of LSTSVM which is simple and fast algorithm but also incorporates the inter-class and intra-class graphs into the model to improve classification accuracy and generalization ability. The experimental results on several synthetic as well as benchmark datasets demonstrate the efficiency of our proposed method. Finally, we further went on to investigate the effectiveness of our classifier for human action recognition application.     

多任务学习的不平衡SVM+算法

处理不平衡数据分类时,传统支持向量机技术（SVM）对少数类样本识别率较低。鉴于SVM+技术能利用样本间隐藏信息的启发,提出了多任务学习的不平衡SVM+算法（MTL-IC-SVM+）。MTL-IC-SVM+基于SVM+将不平衡数据的分类表示为一个多任务的学习问题,并从纠正分类面的偏移出发,分别赋予多数类和少数类样本不同的错分惩罚因子,且设置少数类样本到分类面的距离大于多数类样本到分类面的距离。UCI数据集上的实验结果表明,MTL-IC-SVM+在不平衡数据分类问题上具有较高的分类精度。     

基于模糊双支持向量机的遥感图像分类研究

遥感图像的分类是研究土地利用变化的基础。传统的遥感图像分类方法存在运算速度慢、精度比较低和难以收敛等问题。提出了一种基于模糊双支持向量机的多类分类方法,将模糊技术引入到双支持向量机中,赋予不同样本以不同的模糊隶属度,然后将模糊双支持向量机推广到多类分类中,最后将新方法应用到遥感图像分类中。实验表明,新方法比传统的支持向量机多类分类方法有较高的分类精度,并且有较强的抗噪声能力,在运行时间上也是可行的。模糊双支持向量机是一种有效的遥感图像分类方法。     

基于类向心度的模糊支持向量机

传统支持向量机(SVM)训练含有噪声或野值点的数据时,容易产生过拟合,而模糊支持向量机可以有效地处理这种问题。针对使用样本与类中心之间的距离关系来构建模糊支持向量机隶属度函数的不足,提出了一种基于类向心度的模糊支持向量机(CCD FSVM)。该方法不仅考虑到样本与类中心之间的关系,还考虑到类中各个样本之间的联系,并用类向心度来表示。将类向心度应用于模糊隶属度函数的设计,能够很好地将有效样本与噪声、野值点样本区分开来,而且可以通过向心度的大小,对混合度比较高的样本进行区分,从而达到提高分类精度的效果。实验结果表明,基于类向心度的模糊支持向量机其分类正确率比支持向量机高,在使用三种不同隶属度函数的FSVM中,该方法的抗噪性能最好,分类性能最强。     

基于马氏距离的孪生多分类支持向量机

孪生支持向量机(TWSVM)的研究是近来机器学习领域的一个热点。TWSVM具有分类精度高、训练速度快等优点,但训练时没有充分利用样本的统计信息。作为TWSVM的改进算法,基于马氏距离的孪生支持向量机(TMSVM)在分类过程中考虑了各类样本的协方差信息,在许多实际问题中有着很好的应用效果。然而TMSVM的训练速度有待提高,并且仅适用于二分类问题。针对这两个问题,将最小二乘思想引入TMSVM,用等式约束取代TMSVM中的不等式约束,将二次规划问题的求解简化为求解两个线性方程组,得到基于马氏距离的最小二乘孪生支持向量机(LSTMSVM),并结合有向无环图策略(DAG)设计出基于马氏距离的最小二乘孪生多分类支持向量机。为了减少DAG结构的误差累积,构造了基于马氏距离的类间可分性度量。人工数据集和UCI数据集上的实验均表明,所提算法不仅有效,而且相对于传统多分类SVM,其分类性能有明显提高。     

一种改进的鲁棒模糊孪生支持向量机算法

针对模糊孪生支持向量机算法（FTSVM）对噪声仍然敏感,容易过拟合以及不能有效区分支持向量和离群值等问题,提出了一种改进的鲁棒模糊孪生支持向量机算法（IRFTSVM）。将改进的k近邻隶属度函数和基于类内超平面的隶属度函数结合,构造了一种新的混合隶属度函数;在FTSVM算法的目标函数中引入正则化项和额外的约束条件,实现了结构风险最小化,避免了逆矩阵运算,且非线性问题可以像经典的SVM算法一样直接从线性问题扩展而来;将铰链损失函数替换为pinball损失函数,以此降低对噪声的敏感性。此外,在UCI数据集和人工数据集上对该算法进行评估,并与SVM、TWSVM、FTSVM、PTSVM和TBSVM五个算法进行比较。实验结果表明,该算法的分类结果是令人满意的。     

面向不平衡数据集的改进型SMOTE算法

针对SMOTE（synthetic minority over-sampling technique）在合成少数类新样本时存在的不足,提出了一种改进的SMOTE算法GA-SMOTE。该算法的关键将是遗传算法中的3个基本算子引入到SMOTE中,利用选择算子实现对少数类样本有区别的选择,使用交叉、变异算子实现对合成样本质量的控制.结合GA-SMOTE与SVM（support vector machine）算法来处理不平衡数据的分类问题.UCI数据集上的大量实验表明,GA-SMOTE在新样本的整体合成效果上表现出色,有效提高了SVM在不平衡数据集上的分类性能。     

Fuzzy least squares twin support vector clustering

In this paper, we have formulated a fuzzy least squares version of recently proposed clustering method, namely twin support vector clustering (TWSVC). Here, a fuzzy membership value of each data pattern to different cluster is optimized and is further used for assigning each data pattern to one or other cluster. The formulation leads to finding k cluster center planes by solving modified primal problem of TWSVC, instead of the dual problem usually solved. We show that the solution of the proposed algorithm reduces to solving a series of system of linear equations as opposed to solving series of quadratic programming problems along with system of linear equations as in TWSVC. The experimental results on several publicly available datasets show that the proposed fuzzy least squares twin support vector clustering (F-LS-TWSVC) achieves comparable clustering accuracy to that of TWSVC with comparatively lesser computational time. Further, we have given an application of F-LS-TWSVC for segmentation of color images.

     

Least squares twin support vector machines for pattern classification

In this paper we formulate a least squares version of the recently proposed twin support vector machine (TSVM) for binary classification. This formulation leads to extremely simple and fast algorithm for generating binary classifiers based on two non-parallel hyperplanes. Here we attempt to solve two modified primal problems of TSVM, instead of two dual problems usually solved. We show that the solution of the two modified primal problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems along with two systems of linear equations in TSVM. Classification using nonlinear kernel also leads to systems of linear equations. Our experiments on publicly available datasets indicate that the proposed least squares TSVM has comparable classification accuracy to that of TSVM but with considerably lesser computational time. Since linear least squares TSVM can easily handle large datasets, we further went on to investigate its efficiency for text categorization applications. Computational results demonstrate the effectiveness of the proposed method over linear proximal SVM on all the text corpuses considered.     

Constructing support vector machine ensemble with segmentation for imbalanced datasets

A novel method, namely ensemble support vector machine with segmentation (SeEn–SVM), for the classification of imbalanced datasets is proposed in this paper. In particular, vector quantization algorithm is used to segment the majority class and hence generates some small datasets that are of less imbalance than original one, and two different weighted functions are proposed to integrate all the results of basic classifiers. The goal of the SeEn–SVM algorithm is to improve the prediction accuracy of the minority class, which is more interesting for people. The SeEn–SVM is applied to six UCI datasets, and the results confirmed its better performance than previously proposed methods for imbalance problem.

     

适用于不平衡数据集分类的改进SVM算法

在分析了传统支持向量机(SVM)对不平衡数据的学习缺陷后,提出了一种改进SVM算法,采用自适应合成(ADASYN)采样技术对数据集进行部分重采样,增加少类样本的数量;对不同的样本点分配不同的权重,减弱噪声对训练结果的影响;使用基于代价敏感的SVM算法训练,缓解不平衡数据对超平面造成的偏移.选择UCI数据库中的6组不平衡数据集进行测试,实验结果表明:在各个数据集上改进SVM算法的性能优于其他算法,并在少类准确率和多类准确率上取得了很好的平衡.     

GPU的并行支持向量机算法(英文)

提出了一种新的并行增量式支持向量机算法来解决图形处理单元(GPU)中大规模数据集的分类问题。SVM以及核相关方法可以用来创建精确分类模型,但学习过程需要大量内存和很长时间。扩展了Suykens和Vandewalle提出的最少次方SVM(LS-SVM)方法来建立增量和并行算法。新算法使用图形处理器以低代价获得高系统性能。实现表明,在UCI和Delve数据集上,基于GPU并行增量算法较CPU实现方法快130倍,而且比现行算法,如LibSVM、SVM-perf和CB-SVM等快的多(超过2500倍)。     

核对齐多核模糊支持向量机

支持向量机（SVMs）是当前被广泛使用的机器学习技术,其通过最优分割超平面来提高分类器的泛化能力,在实际应用中表现优异。然而SVM也存在易受噪声影响,以及核函数选择等难题。针对以上问题,本文将基于核对齐的多核学习方法引入到模糊支持向量机（fuzzy support vector machine, FSVM）中,提出了模糊多核支持向量机模型（multiple kernel fuzzy support vector machine,MFSVM）。MFSVM通过模糊粗糙集方法计算每一样例隶属度;其次,利用核对齐的多核方法计算每一单核权重,并将组合核引入到模糊支持向量机中。该方法不仅提高了支持向量机的抗噪声能力,也有效避免了核选择难题。在UCI数据库上进行实验,结果表明本文所提方法具有较高的分类精度,验证了该方法的可行性与有效性。     

基于识别关键样本点的非平衡数据核SVM算法

非平衡数据处理中常采用的欠采样方法很少考虑支持向量机(SVM)的特性,并且在原始空间进行采样会导致多数类样本部分关键信息的丢失.针对上述问题,文中提出基于识别关键样本点的非平衡数据核SVM算法.基于初始超平面有效划分多数类样本,在高维空间中对每个分块进行核异类近邻抽样,得到多数类中的关键样本点,使用关键样本点和少数类样本训练最终核SVM分类器.在多个数据集上的实验证明文中算法的可行性和有效性,特别是在非平衡度高于10∶1的数据集上,文中算法优势明显.     

A new fuzzy support vector machine to evaluate credit risk

Due to recent financial crises and regulatory concerns, financial intermediaries' credit risk assessment is an area of renewed interest in both the academic world and the business community. In this paper, we propose a new fuzzy support vector machine to discriminate good creditors from bad ones. Because in credit scoring areas we usually cannot label one customer as absolutely good who is sure to repay in time, or absolutely bad who will default certainly, our new fuzzy support vector machine treats every sample as both positive and negative classes, but with different memberships. By this way we expect the new fuzzy support vector machine to have more generalization ability, while preserving the merit of insensitive to outliers, as the fuzzy support vector machine (SVM) proposed in previous papers. We reformulate this kind of two-group classification problem into a quadratic programming problem. Empirical tests on three public datasets show that it can have better discriminatory power than the standard support vector machine and the fuzzy support vector machine if appropriate kernel and membership generation method are chosen.