基于超球支持向量机的类增量学习算法研究

提出了一种超球支持向量机类增量学习算法.对每一类样本,利用超球支持向量机在特征空间中求得包围该类尽可能多样本的最小超球,使各类样本之间通过超球隔开.类增量学习过程中,只对新增类样本进行训练,使得该算法在很小的样本集、很小的空间代价下实现了类增量学习,大大降低了训练时间,同时保留了历史训练结果.分类过程中,通过计算待分类样本到各超球球心的距离判定其所属类别,分类简单快捷.实验结果证明,该算法不仅具有较高的训练速度,而且具有较高的分类速度和分类精度.     

基于同心超球面分割的支持向量预抽取方法

支持向量机仅仅由支持向量所决定,因此预先抽取支持向量参与训练是非常重要的。提出了一个基于同心超球面分割的支持向量预抽取方法,并在此基础上给出了HD-SVM训练算法。首先对样本的每一类分别用一些半径足够大的同心超球面进行分割,抽取出距离最优分类面较近的边界样本,这些样本最有可能成为支持向量;然后让边界样本作为初始工作集先参与训练。实验结果表明,该文的方法可以有效地对支持向量进行预抽取,避免了训练全部样本,使得训练速度明显得到提高。     

联合OC-SVM和MC-SVM的图像来源取证方法

为了解决现有图像来源取证方法在相机样本较多时准确性较差、无法对未知模型的图像来源取证以及可扩展性差的问题,提出了一种基于一类和多类支持向量机联合的图像来源取证方法.算法利用协方差的统计相关性提高了CFA插值系数的估计精度,并以SFFS算法选择的特征作为分类器输入.采用OC-SVM(一类支持向量机)和MC-SVM(多类支持向量机)联合的策略进行图像来源分类,有效地解决了对未知模型图像来源的鉴别问题以及可扩展性差的问题.实验表明,该方法对28种相机拍摄的图像进行来源取证,能够达到平均90.4%的鉴别正确率,同时对于3种训练模型以外的未知相机模型拍摄图像,能够达到平均79.3%的检测正确率.     

实现兼类样本增量学习的一种算法

针对兼类样本,提出一种增量学习算法.利用超球支持向量机,在特征空间对属于同一类别的样本求得一个能包围该类尽可能多样本的最小超球,使各类样本之间通过超球隔开.增量学习过程中,只对新增样本以及与新增样本具有相同兼类的旧样本集中的支持向量进行训练,且每次训练只针对一类样本,使得算法在很小的样本集、很小的空间代价下实现兼类样本增量学习,同时保留了与新增样本类别无关的历史训练结果.分类过程中,通过计算待分类样本到各超球球心的距离判定其所属类别,分类准确快捷.实验结果证明了该算法的有效性.     

实现兼类样本类增量学习的一种算法

针对兼类样本,提出一种类增量学习算法.利用超球支持向量机,对每类样本求得一个能包围该类尽可能多样本的最小超球,使各类样本之间通过超球隔开.增量学习时"对新增样本以及旧样本集中的支持向量和超球附近的非支持向量进行训练,使得算法在很小的空间代价下实现兼类样本类增量学习.分类过程中,根据待分类样本到各超球球心的距离判定其所属类别.实验结果表明,该算法具有较快的训练,分类速度和较高的分类精度.

     

实现兼类样本类增量学习的一种算法

针对兼类样本,提出一种类增量学习算法.利用超球支持向量机,对每类样本求得一个能包围该类尽可能多样本的最小超球,使各类样本之间通过超球隔开.增量学习时,对新增样本以及旧样本集中的支持向量和超球附近的非支持向量进行训练,使得算法在很小的空闻代价下实现兼类样本类增量学习.分类过程中,根据待分类样本到各超球球心的距离判定其所属类别.实验结果表明,该算法具有较快的训练、分类速度和较高的分类精度.     

特征选择在隐秘图像检测中的应用

提出一种基于遗传算法和多超球面一类支持向量机的隐秘图像检测方案。为了得到最能反映分类本质的特征从而有效实现分类识别,采用遗传算法进行图像特征选择,将支持向量机的分类效果作为适应度函数值返回,指导遗传算法搜索最优的特征选择方案。实验结果表明,与仅采用支持向量机分类而未进行特征选择的隐秘检测方案相比,该方案提高了隐秘图像检测的识别率。     

超球体多类支持向量机理论

目前的多类分类器大多是经二分类器组合而成的,存在训练速度较慢的问题,在分类类别多的时候,会遇到很大困难,超球体多类支持向量机将超球体单类支持向量机扩展到多类问题,由于每类样本只参与一个超球体支持向量机的训练.因此,这是一种直接多类分类器,训练效率明显提高.为了有效训练超球体多类支持向量机,利用SMO算法思想,提出了超球体支持向量机的快速训练算法.同时对超球体多类支持向量机的推广能力进行了理论上的估计.数值实验表明,在分类类别较多的情况,这种分类器的训练速度有很大提高,非常适合解决类别数较多的分类问题.超球体多类支持向量机为研究快速直接多类分类器提供了新的思路.     

核空间结合样本中心角度的大规模支持向量机

为了提高支持向量机在大规模数据集处理时的精度,提出了基于核空间和样本中心角度的支持向量机算法.在核特征空间下,求得原训练集的两类中心点和两个中心点的超法平面,并获取原训练集样本到超法平面距离和到两中心点中点的比值,用比值最小的n个样本点替代训练集.给出的数学模型显示,该算法不需要计算核空间,比现有的同类缩减策略保留了更多的支持向量数目.结合实例对算法进行了仿真实验,实验结果表明,与同类算法相比,该算法在基本没有降低训练速度的情况下获得了更准确的训练精度.     

基于核函数的支持向量机样本选取算法

使用支持向量机求解大规模数据分类需要较大内存来存储Hessian矩阵,而矩阵的大小则依赖于样本数1,因此在一定程度上导致支持向量机分类效率及质量难以提高.考虑到只有成为支持向量的样本才对决策函数起作用,为了减少训练样本时所需空间及时间开销,提高支持向量机分类效率与质量,提出了一种基于核函数的样本选取算法.该算法通过选取最大可能成为支持向量的样本,以达到减少训练时存储Hessian矩阵所需空间及时间开销的目的.实验结果表明,该算法所筛选出的样本不仅可以提高样本训练准确率,而且可以提高分类计算速度和减少存储空间开销.     

A new maximal-margin spherical-structured multi-class support vector machine

Support vector machines (SVMs), initially proposed for two-class classification problems, have been very successful in pattern recognition problems. For multi-class classification problems, the standard hyperplane-based SVMs are made by constructing and combining several maximal-margin hyperplanes, and each class of data is confined into a certain area constructed by those hyperplanes. Instead of using hyperplanes, hyperspheres that tightly enclosed the data of each class can be used. Since the class-specific hyperspheres are constructed for each class separately, the spherical-structured SVMs can be used to deal with the multi-class classification problem easily. In addition, the center and radius of the class-specific hypersphere characterize the distribution of examples from that class, and may be useful for dealing with imbalance problems. In this paper, we incorporate the concept of maximal margin into the spherical-structured SVMs. Besides, the proposed approach has the advantage of using a new parameter on controlling the number of support vectors. Experimental results show that the proposed method performs well on both artificial and benchmark datasets.     

A Novel and Principled Multiclass Support Vector Machine

Support vector machines (SVMs) have been demonstrated very efficient for binary classification problems; however, computationally efficient and effective multiclass SVMs are still missing. Most existing multiclass SVM classifiers are constructed either by combining multiple binary SVM classifiers, which often perform moderately for some problems, or by converting multiclass problems into one single optimization problem, which is unfortunately computationally expensive. To address these issues, a novel and principled multiclass SVM based on geometric properties of hyperspheres, termed SVMGH, is proposed in this paper. Different from existing SVM‐based methods that seek a cutting hyperplane between two classes, SVMGH draws the discriminative information of each class by constructing a minimum hypersphere containing all class members, and then defines a label function based on the geometric properties of the minimum hyperspheres. We prove theoretically the geometric properties of the minimum hyperspheres to guarantee the validation of SVMGH. The computational efficiency is enhanced by a data reduction strategy as well as a fast training method. Experimental results demonstrate that the proposed SVMGH shows better performance and higher computational efficiency than the state of the art on multiclassification problems while maintaining comparable performance and efficiency on binary classification problems.     

一种基于样本加权的多尺度核支持向量机方法

多核学习方法是机器学习领域中的一个新的热点。核方法通过将数据映射到高维空间来增加线性分类器的计算能力,是目前解决非线性模式分析与分类问题的一种有效途径。但是在一些复杂的情况下,单个核函数构成的核学习方法并不能完全满足如数据异构或者不规则、样本规模大、样本分布不平坦等实际应用中的需求问题,因此将多个核函数进行组合以期获得更好的结果,是一种必然的发展趋势。因此提出一种基于样本加权的多尺度核支持向量机方法,通过不同尺度核函数对样本的拟合能力进行加权,从而得到基于样本加权的多尺度核支持向量机决策函数。通过在多个数据集上的实验分析可以得出所提方法对于各个数据集都获得了很高的分类准确率。     

Generalized discriminant analysis using a kernel approach

We present a new method that we call generalized discriminant analysis (GDA) to deal with nonlinear discriminant analysis using kernel function operator. The underlying theory is close to the support vector machines (SVM) insofar as the GDA method provides a mapping of the input vectors into high-dimensional feature space. In the transformed space, linear properties make it easy to extend and generalize the classical linear discriminant analysis (LDA) to nonlinear discriminant analysis. The formulation is expressed as an eigenvalue problem resolution. Using a different kernel, one can cover a wide class of nonlinearities. For both simulated data and alternate kernels, we give classification results, as well as the shape of the decision function. The results are confirmed using real data to perform seed classification.     

Multiple kernel ensemble learning for software defect prediction

Software defect prediction aims to predict the defect proneness of new software modules with the historical defect data so as to improve the quality of a software system. Software historical defect data has a complicated structure and a marked characteristic of class-imbalance; how to fully analyze and utilize the existing historical defect data and build more precise and effective classifiers has attracted considerable researchers’ interest from both academia and industry. Multiple kernel learning and ensemble learning are effective techniques in the field of machine learning. Multiple kernel learning can map the historical defect data to a higher-dimensional feature space and make them express better, and ensemble learning can use a series of weak classifiers to reduce the bias generated by the majority class and obtain better predictive performance. In this paper, we propose to use the multiple kernel learning to predict software defect. By using the characteristics of the metrics mined from the open source software, we get a multiple kernel classifier through ensemble learning method, which has the advantages of both multiple kernel learning and ensemble learning. We thus propose a multiple kernel ensemble learning (MKEL) approach for software defect classification and prediction. Considering the cost of risk in software defect prediction, we design a new sample weight vector updating strategy to reduce the cost of risk caused by misclassifying defective modules as non-defective ones. We employ the widely used NASA MDP datasets as test data to evaluate the performance of all compared methods; experimental results show that MKEL outperforms several representative state-of-the-art defect prediction methods.     

一种新的核化SVM多层分类方法

利用核化思想提出了一种新的SVM多层分类算法。该算法的基本思路是：先利用Mercer核,将输入空间非线性可分的训练样本映射到高维特征空间Hilbert中,使之线性可分,然后采用最小超球体类包含作为层次分类的依据来生成二叉决策树,从而实现在高维空间中的多类分类。实验表明,采用该算法进行多类分类,可以有效地解决输入空间非线性可分问题,并可在一定程度上提高分类器的分类精度。     

Optimizing the kernel in the empirical feature space

In this paper, we present a method of kernel optimization by maximizing a measure of class separability in the empirical feature space, an Euclidean space in which the training data are embedded in such a way that the geometrical structure of the data in the feature space is preserved. Employing a data-dependent kernel, we derive an effective kernel optimization algorithm that maximizes the class separability of the data in the empirical feature space. It is shown that there exists a close relationship between the class separability measure introduced here and the alignment measure defined recently by Cristianini. Extensive simulations are carried out which show that the optimized kernel is more adaptive to the input data, and leads to a substantial, sometimes significant, improvement in the performance of various data classification algorithms.     

基于Bhattacharyya距离准则的核空间特征提取算法

提出了一种新的以Bhattacharyya距离为准则的核空间特征提取算法．该算法的核心思想是把样本非线性映射到高维核空间．在核空间中寻找一组最优特征向量，然后把样本线性映射到低维特征空间，使类别间的Bhattacharyya距离最大。从而保证Bayes分类误差上界最小．采用核函数技术，把特征提取问题转化为一个QP(Quadratic Programming)优化问题．保证了算法的全局收敛性和快速性．此算法具有两个优点：(1)该算法提取的特征对数据分类来说更有效；(2)对于给定的模式分类问题，算法可以预测出在不损失分类精度情况下所必须的特征向量数目的上界，并能够提取出分类有效特征．实验结果表明，该算法的性能与理论分析的结论相吻合，优于目前常用的特征提取算法．     

基于支持向量机的不平衡数据分类算法的研究*

针对不平衡数据分类问题,提出了基于Smote与核函数修改相结合的算法。首先用Smote方法处理数 据,降低不平衡度;然后以黎曼几何为依据,利用保角变换,对核函数进行修改,提高支持向量机的分类泛化能 力;最后用修改后的支持向量机对新的数据进行处理。实验结果表明,这种方法能在保持整体正确率的前提下 有效地提高少数类样本的分类准确率。     

Low-Rank Bilinear Classification: Efficient Convex Optimization and Extensions

In pattern classification, it is needed to efficiently treat not only feature vectors but also feature matrices defined as two-way data, while preserving the two-way structure such as spatio-temporal relationships. The classifier for the feature matrix is generally formulated in a bilinear form composed of row and column weights which jointly result in a matrix weight. The rank of the matrix should be low from the viewpoint of generalization performance and computational cost. For that purpose, we propose a low-rank bilinear classifier based on the efficient convex optimization. In the proposed method, the classifier is optimized by minimizing the trace norm of the classifier (matrix) to reduce the rank without any hard constraint on it. We formulate the optimization problem in a tractable convex form and provide the procedure to solve it efficiently with the global optimum. In addition, we propose two novel extensions of the bilinear classifier in terms of multiple kernel learning and cross-modal learning. Through kernelizing the bilinear method, we naturally induce a novel multiple kernel learning. The method integrates both the inter kernels between heterogeneous reproducing kernel Hilbert spaces (RKHSs) and the ordinary kernels within respective RKHSs into a new discriminative kernel in a unified manner using the bilinear model. Besides, for cross-modal learning, we consider to map into the common space the multi-modal features which are subsequently classified in that space. We show that the projection and the classification are jointly represented by the bilinear model, and then propose the method to optimize both of them simultaneously in the bilinear framework. In the experiments on various visual classification tasks, the proposed methods exhibit favorable performances compared to the other methods.