基于相似度差的大间隔快速学习模型

许多模式分类方法比如支持向量机和L₂核分类器等都会利用核方法并转化为二次规划问题进行求解,而计算核矩阵需要O(m²)的空间复杂度,求解QP问题则需要O(m³)的时间复杂度,这就使得此类方法在大样本数据上的学习性能非常低下。对此,首次提出了相似度差支持向量机算法DSSVM。算法旨在寻求样本与某类相似度的一个最佳线性表示,并从线性表示的稀疏性以及相似度差意义上的间隔最大化角度构造了新的最优化问题。同时,证明了该算法等价于中心约束型最小包含球问题,这样就可以通过引入最小包含球的快速学习理论将相似度差支持向量机扩展为相似度差核支持向量机DSCVM,从而较好地解决了大规模数据集的分类问题。实验证明了相似度差支持向量机和相似度差核支持向量机的有效性。     

基于模糊时序和支持向量机的高速公路SO2浓度预测算法

针对现有SO₂浓度预测方法中存在的污染物来源和影响因素认识不统一、小样本数据敏感、易于陷入局部最优等问题,文中提出了基于模糊时序和支持向量机的高速公路SO₂浓度预测算法,为搭建高速公路环境健康监测系统提供了可靠的理论支持.该方法依据SO₂浓度的季节变动规律,以季节作为时间序列,以24h为粒化窗宽,通过高斯核函数提取原始样本数据的特征值,输入支持向量机训练模型,并利用k重交叉验证法结合网格划分优化模型参数.文中应用该方法建立了SO₂浓度预测模型,并以2014年4月至2015年3月山西省太旧高速公路某监测点SO₂小时浓度监测值为样本数据,在MATLAB平台下应用LIBSVM工具实现了计算过程.结果表明,基于模糊时序和支持向量机的高速公路SO₂浓度预测算法不受机理性理论研究的限制,支持小样本学习,非线性拟合效果好,泛化能力强.     

一族三次样条光滑半监督支持向量机

针对半监督支持向量分类优化中的非凸非光滑化问题,建立光滑半监督支持向量机模型,提出基于分段多项式函数和插值思想构造一个新的三次样条光滑函数,从而可以更好地逼近对半监督支持向量机中非光滑的对称铰链损失函数部分,构造出基于此光滑函数的具有二阶光滑的半监督支持向量机模型。进而可以用优化中的光滑算法来求解该模型,并分析所构造的三次样条函数对对称铰链损失函数的逼近精度。通过数据实验证明所构造的新的光滑半监督模型具有较好的分类效果和效率。     

基于- 范数约束的LSSVR 多核学习算法

针对核函数选择对最小二乘支持向量机回归模型泛化性的影响, 提出一种新的基于????- 范数约束的最小二乘支持向量机多核学习算法. 该算法提供了两种求解方法, 均通过两重循环进行求解, 外循环用于更新核函数的权值, 内循环用于求解最小二乘支持向量机的拉格朗日乘数, 充分利用该多核学习算法, 有效提高了最小二乘支持向量机的泛化能力, 而且对惩罚参数的选择具有较强的鲁棒性. 基于单变量和多变量函数的仿真实验表明了所提出算法的有效性.

     

分段光滑的半监督支持向量分类机

为了解决半监督支持向量分类优化模型中的非凸非光滑问题,基于分段逼近的思想提出了一个分段函数,并以此逼近非凸非光滑的目标函数。给出的分段函数可以根据不同的精度要求选择不同的逼近参数,同时构造出基于上述分段函数的光滑半监督支持向量机模型。采用了LDS(Low Density Separation)算法求解模型,分析了其对对称铰链损失函数的逼进精度。理论分析和数值实验结果都证明分段光滑的半监督支持向量机的分类性能和效率优于以往提出的光滑模型。     

一种多项式光滑的半监督支持向量机分类算法

为了处理半监督支持向量机优化中的非凸非光滑问题,引入一个多项式光滑函数来逼近非凸的目标函数,给出的多项式函数在样本的高密度区逼近精度高,逼近精度低时出现在样本的低密度区.采用共轭梯度法求解模型.在人工数据和UCI数据库中的4个数据集上的实验结果显示,算法不仅能保证标号数据很少时的分类精度,而且不因标号数据的增多而明显提高分类性能,因此给出的分类器性能是稳定的.     

一种半监督支持向量机优化方法

针对半监督支持向量机在采用间隔最大化思想对有标签样本和无标签样本进行分类时面临的非凸优化问题,提出了一种采用分布估计算法进行半监督支持向量机优化的方法EDA_S3VM。该方法把无标签样本的标签作为需要优化的参数,从而得到一个在标准支持向量机上的组合优化问题,利用分布估计算法通过概率模型的学习和采样来对问题进行求解。在人工数据集和公共数据集上的实验结果表明,EDA_S3VM与其它一些半监督支持向量机算法相比有更高的分类准确率。     

基于支持向量机协同训练的半监督回归

将支持向量机与半监督学习理论相结合,提出基于支持向量机协同训练的半监督回归模型,使用两个支持向量机回归模型相互影响,协同训练。利用实验数据集进行实验,并与监督支持向量机回归模型、半监督自训练支持向量机回归模型作比较。实验结果表明,基于支持向量机协同训练的半监督回归模型在缺少标记样本的情况下,提高了回归估计的精度。     

基于相关向量机的非线性动态系统辨识

基于具有核函数不用满足Mercer条件、相关向鼍自动确定及核函数少特点的稀疏贝叶斯的相关向量机核学习方法,提出了平滑先验条件约束的相关向量机的学习方法,采用稀疏贝叶斯模型的最大边缘似然算法加快了求解相关向量机的向量,并采取交叉验证法确定其核参数提高了相关向量机辨识的泛化性.该方法避免了支持向量机的非线性系统辨识的模型结构难于确定的问题,与支持向量机辨识方法相比较,辨识的模型结构更简洁.仿真表明,该方法应用于非线性动态系统的辨识,具有良好的效果.     

基于半监督支持向量机的交互式遗传算法

为减轻用户疲劳并将交互式遗传算法应用于复杂的优化问题中,提出一种基于半监督支持向量机的交互式遗传算法。根据标记样本和未标记样本几何特性派生出数据依赖的核函数,以此构建半监督支持向量机,再以自训练方法进行高可信未标记样本的批量选择,实现用户评价代理模型的高泛化性能。将该方法应用于基于内容的图像检索系统,结果表明其能有效加快进化收敛的速度,提高优化成功率。     

基于K均值聚类和粒子群优化的多核SVM图像分割

图像分割是图像理解和计算机视觉的重要内容.针对单核SVM在进行图像分割过程中不能兼顾分割精度高和泛化性能好的问题,提出一种基于K均值聚类和优化多核SVM的图像分割算法.该算法首先运用K均值聚类算法自动选取训练样本,然后提取其颜色特征和纹理特征作为训练样本的特征属性,并使用其对构造的多核SVM分割模型进行训练,最后用粒子群优化算法对多核核参数、惩罚因子以及核权重系数联合寻优,使生成的多核SVM具有更好的分割性能.实验结果表明,本文方法在有效提取图像目标细节的同时,获得了更高的分割精度,与基于单核的SVM分割模型相比,具有更强的泛化能力.     

Evolution strategies based adaptive Lp LS-SVM

Not only different databases but two classes of data within a database can also have different data structures. SVM and LS-SVM typically minimize the empirical ?-risk; regularized versions subject to fixed penalty (L₂ or L₁ penalty) are non-adaptive since their penalty forms are pre-determined. They often perform well only for certain types of situations. For example, LS-SVM with L₂ penalty is not preferred if the underlying model is sparse. This paper proposes an adaptive penalty learning procedure called evolution strategies (ES) based adaptive L_p least squares support vector machine (ES-based L_p LS-SVM) to address the above issue. By introducing multiple kernels, a L_p penalty based nonlinear objective function is derived. The iterative re-weighted minimal solver (IRMS) algorithm is used to solve the nonlinear function. Then evolution strategies (ES) is used to solve the multi-parameters optimization problem. Penalty parameterp, kernel and regularized parameters are adaptively selected by the proposed ES-based algorithm in the process of training the data, which makes it easier to achieve the optimal solution. Numerical experiments are conducted on two artificial data sets and six real world data sets. The experiment results show that the proposed procedure offer better generalization performance than the standard SVM, the LS-SVM and other improved algorithms.     

New Least Squares Support Vector Machines Based on Matrix Patterns

Support vector machine (SVM), as an effective method in classification problems, tries to find the optimal hyperplane that maximizes the margin between two classes and can be obtained by solving a constrained optimization criterion using quadratic programming (QP). This QP leads to higher computational cost. Least squares support vector machine (LS-SVM), as a variant of SVM, tries to avoid the above shortcoming and obtain an analytical solution directly from solving a set of linear equations instead of QP. Both SVM and LS-SVM operate directly on patterns represented by vector, i.e., before applying SVM or LS-SVM to a pattern, any non-vector pattern such as an image has to be first vectorized into a vector pattern by some techniques like concatenation. However, some implicit structural or local contextual information may be lost in this transformation. Moreover, as the dimension d of the weight vector in SVM or LS-SVM with the linear kernel is equal to the dimension d ₁ × d ₂ of the original input pattern, as a result, the higher the dimension of a vector pattern is, the more space is needed for storing it. In this paper, inspired by the method of feature extraction directly based on matrix patterns and the advantages of LS-SVM, we propose a new classifier design method based on matrix patterns, called MatLSSVM, such that the new method can not only directly operate on original matrix patterns, but also efficiently reduce memory for the weight vector (d) from d ₁ × d ₂ to d ₁ + d ₂. However like LS-SVM, MatLSSVM inherits LS-SVM’s existence of unclassifiable regions when extended to multi-class problems. Thus with the fuzzy version of LS-SVM, a corresponding fuzzy version of MatLSSVM (MatFLSSVM) is further proposed to remove unclassifiable regions effectively for multi-class problems. Experimental results on some benchmark datasets show that the proposed method is competitive in classification performance compared to LS-SVM, fuzzy LS-SVM (FLS-SVM), more-recent MatPCA and MatFLDA. In addition, more importantly, the idea used here has a possibility of providing a novel way of constructing learning model.     

A Primal Framework for Indefinite Kernel Learning

Kernel methods have been widely applied in machine learning to solve complex nonlinear problems. Kernel selection is one of the key issues in kernel methods, since it is vital for improving generalization performance. Traditionally, the selection of kernel is restricted to be positive definite which makes their applicability partially limited. Actually, in many real applications such as gene identification and object recognition, indefinite kernels frequently emerge and can achieve better performance. However, compared to positive definite ones, indefinite kernels are more complicated due to the non-convexity of the subsequent optimization problems, which leads to the incapability of most existing kernel algorithms. Some indefinite kernel methods have been proposed based on the dual of support vector machine (SVM), which mostly emphasize on how to transform the non-convex optimization to be convex by using positive definite kernels to approximate indefinite ones. In fact, the duality gap in SVM usually exists in the case of indefinite kernels and therefore these algorithms do not indeed solve the indefinite kernel problems themselves. In this paper, we present a novel framework for indefinite kernel learning derived directly from the primal of SVM, which establishes several new models not only for single indefinite kernel but also extends to multiple indefinite kernel scenarios. Several algorithms are developed to handle the non-convex optimization problems in these models. We further provide a constructive approach for kernel selection in the algorithms by using the theory of similarity functions. Experiments on real world datasets demonstrate the superiority of our models.     

结合半监督核的高斯过程分类

提出了一种半监督算法用于学习高斯过程分类器, 其通过结合非参数的半监督核向分类器提供未标记数据信息. 该算法主要包括以下几个方面: 1)通过图拉普拉斯的谱分解获得核矩阵, 其联合了标记数据和未标记数据信息; 2)采用凸最优化方法学习核矩阵特征向量的最优权值, 构建非参数的半监督核; 3)把半监督核整合到高斯过程模型中, 构建所提出的半监督学习算法. 该算法的主要特点是: 把基于整个数据集的非参数半监督核应用于高斯过程模型, 该模型有着明确的概率描述, 可以方便地对数据之间的不确定性进行建模, 并能够解决复杂的推论问题. 通过实验结果表明, 该算法与其他方法相比具有更高的可靠性.     

基于CPSO的混合核函数SVM参数优化及应用

支持向量机(SVM)建模的拟合精度和泛化能力取决于相关参数的选取,目前SVM中的参数的寻优一般只针对惩罚系数和核参数,而混合核函数的引入,使SVM增加了一个可调参数.针对混合核函数SVM的多参数选择问题,提出利用具有较强全局搜索能力的混沌粒子群(CPSO)优化算法对混合核函数SVM建模过程中的重要参数进行优化调整,每一...     

核对齐多核模糊支持向量机

支持向量机（SVMs）是当前被广泛使用的机器学习技术,其通过最优分割超平面来提高分类器的泛化能力,在实际应用中表现优异。然而SVM也存在易受噪声影响,以及核函数选择等难题。针对以上问题,本文将基于核对齐的多核学习方法引入到模糊支持向量机（fuzzy support vector machine, FSVM）中,提出了模糊多核支持向量机模型（multiple kernel fuzzy support vector machine,MFSVM）。MFSVM通过模糊粗糙集方法计算每一样例隶属度;其次,利用核对齐的多核方法计算每一单核权重,并将组合核引入到模糊支持向量机中。该方法不仅提高了支持向量机的抗噪声能力,也有效避免了核选择难题。在UCI数据库上进行实验,结果表明本文所提方法具有较高的分类精度,验证了该方法的可行性与有效性。     

Least Square Transduction Support Vector Machine

Support vector machine (SVM) is a general and powerful learning machine, which adopts supervised manner. However, for many practical machine learning and data mining applications, unlabeled training examples are readily available but labeled ones are very expensive to be obtained. Therefore, semi-supervised learning emerges as the times require. At present, the combination of SVM and semi-supervised learning principle such as transductive learning has attracted more and more attentions. Transductive support vector machine (TSVM) learns a large margin hyperplane classifier using labeled training data, but simultaneously force this hyperplane to be far away from the unlabeled data. TSVM might seem to be the perfect semi-supervised algorithm since it combines the powerful regularization of SVMs and a direct implementation of the clustering assumption, nevertheless its objective function is non-convex and then it is difficult to be optimized. This paper aims to solve this difficult problem. We apply least square support vector machine to implement TSVM, which can ensure that the objective function is convex and the optimization solution can then be easily found by solving a set of linear equations. Simulation results demonstrate that the proposed method can exploit unlabeled data to yield good performance effectively.