Using Sequential Unconstrained Minimization Techniques to simplify SVM solvers

In this paper, we apply Sequential Unconstrained Minimization Techniques (SUMTs) to the classical formulations of both the classical L1 norm SVM and the least squares SVM. We show that each can be solved as a sequence of unconstrained optimization problems with only box constraints. We propose relaxed SVM and relaxed LSSVM formulations that correspond to a single problem in the corresponding SUMT sequence. We also propose a SMO like algorithm to solve the relaxed formulations that works by updating individual Lagrange multipliers. The methods yield comparable or better results on large benchmark datasets than classical SVM and LSSVM formulations, at substantially higher speeds.     

Using SVM based method for equipment fault detection in a thermal power plant

Due to the growing demand on electricity, how to improve the efficiency of equipment in a thermal power plant has become one of the critical issues. Reports indicate that efficiency and availability are heavily dependant upon high reliability and maintainability. Recently, the concept of e-maintenance has been introduced to reduce the cost of maintenance. In e-maintenance systems, the intelligent fault detection system plays a crucial role for identifying failures. Data mining techniques are at the core of such intelligent systems and can greatly influence their performance. Applying these techniques to fault detection makes it possible to shorten shutdown maintenance and thus increase the capacity utilization rates of equipment. Therefore, this work proposes a support vector machines (SVM) based model which integrates a dimension reduction scheme to analyze the failures of turbines in thermal power facilities. Finally, a real case from a thermal power plant is provided to evaluate the effectiveness of the proposed SVM based model. Experimental results show that SVM outperforms linear discriminant analysis (LDA) and back-propagation neural networks (BPN) in classification performance.     

Polynomial kernel adaptation and extensions to the SVM classifier learning

Three extensions to the Kernel-AdaTron training algorithm for Support Vector Machine classifier learning are presented. These extensions allow the trained classifier to adhere more closely to the constraints imposed by Support Vector Machine theory. The results of these modifications show improvements over the existing Kernel-AdaTron algorithm. A method of parameter optimisation for polynomial kernels is also proposed.     

基于混沌优化算法的支持向量机参数选取方法

支持向量机（SVM）的参数取值决定了其学习性能和泛化能力．对此,将SVM参数的选取看作参数的组合优化,建立组合优化的目标函数,采用变尺度混沌优化算法来搜索最优目标函数值．混沌优化算法是一种全局搜索方法,在选取SVM参数时,不必考虑模型的复杂度和变量维数．仿真表明,混沌优化算法是选取SVM参数的有效方法,应用到函数逼近时具有优良的性能．     

GSVM: An SVM for handling imbalanced accuracy between classes inbi-classification problems

A new support vector machine, SVM, is introduced, called GSVM, which is specially designed for bi-classification problems where balanced accuracy between classes is the objective. Starting from a standard SVM, the GSVM is obtained from a low-cost post-processing strategy by modifying the initial bias. Thus, the bias for GSVM is calculated by moving the original bias in the SVM to improve the geometric mean between the true positive rate and the true negative rate. The proposed solution neither modifies the original optimization problem for SVM training, nor introduces new hyper-parameters. Experimentation carried out on a high number of databases (23) shows GSVM obtaining the desired balanced accuracy between classes. Furthermore, its performance improves well-known cost-sensitive schemes for SVM, without adding complexity or computational cost.     

基于Matlab的支持向量机工具箱

介绍了基于MATLAB的支持向量机工具箱,详细说明了工具箱中用于支持向量分类和支持向量回归的函数.并通过两个具体的实例来说明利用SVM工具箱进行分类和回归方面的方法.     

Privacy-preserving SVM classification

Traditional Data Mining and Knowledge Discovery algorithms assume free access to data, either at a centralized location or in federated form. Increasingly, privacy and security concerns restrict this access, thus derailing data mining projects. What is required is distributed knowledge discovery that is sensitive to this problem. The key is to obtain valid results, while providing guarantees on the nondisclosure of data. Support vector machine classification is one of the most widely used classification methodologies in data mining and machine learning. It is based on solid theoretical foundations and has wide practical application. This paper proposes a privacy-preserving solution for support vector machine (SVM) classification, PP-SVM for short. Our solution constructs the global SVM classification model from data distributed at multiple parties, without disclosing the data of each party to others. Solutions are sketched out for data that is vertically, horizontally, or even arbitrarily partitioned. We quantify the security and efficiency of the proposed method, and highlight future challenges. Jaideep Vaidya received the Bachelor’s degree in Computer Engineering from the University of Mumbai. He received the Master’s and the Ph.D. degrees in Computer Science from Purdue University. He is an Assistant Professor in the Management Science and Information Systems Department at Rutgers University. His research interests include data mining and analysis, information security, and privacy. He has received best paper awards for papers in ICDE and SIDKDD. He is a Member of the IEEE Computer Society and the ACM. Hwanjo Yu received the Ph.D. degree in Computer Science in 2004 from the University of Illinois at Urbana-Champaign. He is an Assistant Professor in the Department of Computer Science at the University of Iowa. His research interests include data mining, machine learning, database, and information systems. He is an Associate Editor of Neurocomputing and served on the NSF Panel in 2006. He has served on the program committees of 2005 ACM SAC on Data Mining track, 2005 and 2006 IEEE ICDM, 2006 ACM CIKM, and 2006 SIAM Data Mining. Xiaoqian Jiang received the B.S. degree in Computer Science from Shanghai Maritime University, Shanghai, 2003. He received the M.C.S. degree in Computer Science from the University of Iowa, Iowa City, 2005. Currently, he is pursuing a Ph.D. degree from the School of Computer Science, Carnegie Mellon University. His research interests are computer vision, machine learning, data mining, and privacy protection technologies.     

Active Learning for Transductive Support Vector Machines with Applications to Text Classification

This paper presents a novel active learning approach for transductive support vector machines with applications to text classification. The concept of the centroid of the support vectors is proposed so that the selective sampling based on measuring the distance from the unlabeled samples to the centroid is feasible and simple to compute. With additional hypothesis, active learning offers better performance with comparison to regular inductive SVMs and transductive SVMs with random sampling,and it is even competitive to transductive SVMs on all available training data. Experimental results prove that our approach is efficient and easy to implement.     

Comprehensive evaluation of twin SVM based classifiers on UCI datasets

In the past decade, twin support vector machine (TWSVM) based classifiers have received considerable attention from the research community. In this paper, we analyze the performance of 8 variants of TWSVM based classifiers along with 179 classifiers evaluated in Fernandez-Delgado et al. (2014) from 17 different families on 90 University of California Irvine (UCI) benchmark datasets from various domains. Results of these classifiers are exhaustively analyzed using various performance criteria. Statistical testing is performed using Friedman Rank (FRank). Our experiments show that two least square TWSVM based classifiers (ILSTSVM_m, and RELS-TSVM_m) are the top two ranked methods among 187 classifiers and they significantly outperform all other classifiers according to Friedman Rank. Overall, this paper bridges the evaluational benchmarking gap between various TWSVM variants and the classifiers from other families. Codes of this paper are provided on authors’ homepages to reproduce the presented results and figures in this paper.     

多项式核函数SVM快速分类算法

标准的SVM分类计算过程中有大量的支持向量参与了计算,导致了分类速度缓慢。该文为提高SVM的分类速度,提出了一种快速的多项式核函数SVM分类算法,即将使用多项式核的SVM分类决策函数展开为关于待分类向量各分量的多项式,分类时通过计算各个多项式的值而得到分类结果,使分类计算量和支持向量数量无关,又保留了全部支持向量的信息。当多项式核函数的阶数或待分类向量的维数较低而支持向量数量较多时,使用该算法可以使SVM 分类的速度得到极大的提高。针对实际数据集的实验表明了该算法的有效性。     

一种LDA与SVM混合的多类分类方法

针对决策有向无环图支持向量机（ＤＤＡＧＳＶＭ）需训练大量支持向量机（ＳＶＭ）和误差积累的问题,提出一种线性判别分析（ＬＤＡ）与ＳＶＭ 混合的多类分类算法．首先根据高维样本在低维空间中投影的特点,给出一种优化ＬＤＡ 分类阈值;然后以优化ＬＤＡ 对每个二类问题的分类误差作为类间线性可分度,对线性可分度较低的问题采用非线性ＳＶＭ 加以解决,并以分类误差作为对应二类问题的可分度;最后将可分度作为混合ＤＤＡＧ 分类器的决策依据．实验表明,与ＤＤＡＧＳＶＭ 相比,所提出算法在确保泛化精度的条件下具有更高的训练和分类速度．     

WLS-SVM算法用于DCSK通信系统降噪

本文提出了基于加权最小二乘支撑矢量机（WLS-SVM）学习算法的一种DCSK混沌通信系统降噪方法。给定接收信号为训练样本集,首先用最小二乘支撑矢量机（LS-SVM）对样本数据进行估计得到估计误差,根据估计误差的统计分布特性获得一个加权系数,然后再求解WLS-SVM,得到优化的接收信号的估计值,达到降噪的目的。仿真结果表明,优化后的系统误码率（BER）性能与DCSK系统的理论噪声性能相比得到改善。     

A hybrid SVM based decision tree

We have proposed a hybrid SVM based decision tree to speedup SVMs in its testing phase for binary classification tasks. While most existing methods addressed towards this task aim at reducing the number of support vectors, we have focused on reducing the number of test datapoints that need SVM’s help in getting classified. The central idea is to approximate the decision boundary of SVM using decision trees. The resulting tree is a hybrid tree in the sense that it has both univariate and multivariate (SVM) nodes. The hybrid tree takes SVM’s help only in classifying crucial datapoints lying near decision boundary; remaining less crucial datapoints are classified by fast univariate nodes. The classification accuracy of the hybrid tree is guaranteed by tuning a threshold parameter. Extensive computational comparisons on 19 publicly available datasets indicate that the proposed method achieves significant speedup when compared to SVMs, without any compromise in classification accuracy.     

基于加权支持向量机的移动建模方法及其在软测量中的应用

工业过程软测量技术的核心问题是建立软测量模型,然而,利用传统全局建模方法与多模型建模方法进行复杂工业过程软测量建模时,在不同程度上存在一些问题.本文利用支持向量机(SVMs)泛化能力强的特点,结合局部加权学习(LWL)算法思想,提出一种适于局部学习的加权支持向量机(W_SVMs)学习算法和基于这种算法的移动建模方法.利用这种建模方法对Box-Jenkins煤气炉和重油催化裂化(FCCU)装置进行分析建模,并与其它不同建模方法进行比较,显示了该方法的优点和有效性.     

一种结合波段分组特征和形态学特征的高光谱图像分类方法

如何准确识别图像中的类别信息,是计算机视觉和模式识别领域的重要研究问题。遥感卫星图像数据,尤其是高光谱等遥感图像数据的出现,将空间信息与光谱信息集成于同一数据集中,丰富了图像信息来源。如何准确地识别高光谱图像中的地物类别,已经成为了图像处理和模式识别领域的热点问题。面向高光谱图像数据提出了一种基于波段分组特征和形态学特征的高光谱图像分类方法,结合空间和光谱特征提高分类精度。通过真实的高光谱数据实验证明:利用波段分组可以有效地保持光谱特征,降低数据冗余;在波段分组基础上结合形态学特征进行分类,比传统分类方法的分类精度明显提高。     

Stray Example Sheltering by Loss Regularized SVM and <Emphasis Type="Italic">k</Emphasis>NN Preprocessor

This paper presents a new model developed by merging a non-parametric k-nearest-neighbor (kNN) preprocessor into an underlying support vector machine (SVM) to provide shelters for meaningful training examples, especially for stray examples scattered around their counterpart examples with different class labels. Motivated by the method of adding heavier penalty to the stray example to attain a stricter loss function for optimization, the model acts to shelter stray examples. The model consists of a filtering kNN emphasizer stage and a classical classification stage. First, the filtering kNN emphasizer stage was employed to collect information from the training examples and to produce arbitrary weights for stray examples. Then, an underlying SVM with parameterized real-valued class labels was employed to carry those weights, representing various emphasized levels of the examples, in the classification. The emphasized weights given as heavier penalties changed the regularization in the quadratic programming of the SVM, and brought the resultant decision function into a higher training accuracy. The novel idea of real-valued class labels for conveying the emphasized weights provides an effective way to pursue the solution of the classification inspired by the additional information. The adoption of the kNN preprocessor as a filtering stage is effective since it is independent of SVM in the classification stage. Due to its property of estimating density locally, the kNN method has the advantage of distinguishing stray examples from regular examples by merely considering their circumstances in the input space. In this paper, detailed experimental results and a simulated application are given to address the corresponding properties. The results show that the model is promising in terms of its original expectations.     

SVM用于基于块划分特征提取的图像分类

在基于内容图像检索中,图像的底层视觉特征和高层语义概念之间存在着较大的语义间隔。使用机器学习方法学习图像特征,自动建立图像类的模型成为一种有效的方法。本文提出了一种用支持向量机(SVM)实现自然图像自动语义归类的方法,基于块划分聚类得到特征向量作为SVM训练样本,实现语义分类器。由于参与聚类的是某类图像所有块的特征,提取的特征更能反映某一类图像特征。实验证明这种方法是有效的。     

Feature selection via sensitivity analysis of SVM probabilistic outputs

Feature selection is an important aspect of solving data-mining and machine-learning problems. This paper proposes a feature-selection method for the Support Vector Machine (SVM) learning. Like most feature-selection methods, the proposed method ranks all features in decreasing order of importance so that more relevant features can be identified. It uses a novel criterion based on the probabilistic outputs of SVM. This criterion, termed Feature-based Sensitivity of Posterior Probabilities (FSPP), evaluates the importance of a specific feature by computing the aggregate value, over the feature space, of the absolute difference of the probabilistic outputs of SVM with and without the feature. The exact form of this criterion is not easily computable and approximation is needed. Four approximations, FSPP1-FSPP4, are proposed for this purpose. The first two approximations evaluate the criterion by randomly permuting the values of the feature among samples of the training data. They differ in their choices of the mapping function from standard SVM output to its probabilistic output: FSPP1 uses a simple threshold function while FSPP2 uses a sigmoid function. The second two directly approximate the criterion but differ in the smoothness assumptions of criterion with respect to the features. The performance of these approximations, used in an overall feature-selection scheme, is then evaluated on various artificial problems and real-world problems, including datasets from the recent Neural Information Processing Systems (NIPS) feature selection competition. FSPP1-3 show good performance consistently with FSPP2 being the best overall by a slight margin. The performance of FSPP2 is competitive with some of the best performing feature-selection methods in the literature on the datasets that we have tested. Its associated computations are modest and hence it is suitable as a feature-selection method for SVM applications. Editor: Risto Miikkulainen.     

一种基于免疫算子的SVM算法

SVM是一种基于核函数的机器学习算法,因为它具有良好的推广性和较好的性能,所以成为近些年来大家所关注的热点,但是该算法存在两个问题：一、如何提高SVM的计算精度;二、如何减少计算时间。本文提出一种使用免疫算子的SVM算法,该算法不但能够提高SVM的性能使其更加接近于实际问题,还能避免因问题太复杂使得结果不是最优解的情况。文中最后对样本进行了实验,结果说明了使用免疫算子的方法比经典方法在分类效果上有明显提高。     

基于小波和支持向量机的人脸识别技术

小波变换具有良好的多尺度特征表达能力,能将图像的大部分能量集中到最低分辨率子图像,高频部分则对应于图像的边缘和轮廓,可以很好地压缩和表征人脸图像的特征。支持向量机技术针对小样本问题设计,对人脸识别这样的非线性、高维数的小样本问题有非常好的分类效果和学习推广能力,目前已经成为模式识别的首选分类器。文中使用小波变换来对人脸的高维图像矢量进行压缩,并设计了一个支持向量机分类器系统来识别人脸。试验结果验证了该系统有很高的识别率和较强的鲁棒性。