Model selection for the LS-SVM. Application to handwriting recognition

The support vector machine (SVM) is a powerful classifier which has been used successfully in many pattern recognition problems. It has also been shown to perform well in the handwriting recognition field. The least squares SVM (LS-SVM), like the SVM, is based on the margin-maximization principle performing structural risk minimization. However, it is easier to train than the SVM, as it requires only the solution to a convex linear problem, and not a quadratic problem as in the SVM. In this paper, we propose to conduct model selection for the LS-SVM using an empirical error criterion. Experiments on handwritten character recognition show the usefulness of this classifier and demonstrate that model selection improves the generalization performance of the LS-SVM.     

一种新的软间隔支持向量机分类算法

软间隔支持向量机（SVM）分类算法是目前最具有代表性的模式分类算法之一,它在应用中的一个主要困难是确定控制参数C.提出一种新的软间隔SVM分类算法,通过松弛变量改变约束条件,允许数据点进入分离区域但不越过分类超平面,从而避免了参数C的确定问题.计算机实验和故障诊断实例表明,基于新算法的SVM分类器有较高的分类准确性和较好的泛化性能,能够实际应用于模式分类.     

IP-LSSVM: A two-step sparse classifier

We present in this work a two-step sparse classifier called IP-LSSVM which is based on Least Squares Support Vector Machine (LS-SVM). The formulation of LS-SVM aims at solving the learning problem with a system of linear equations. Although this solution is simpler, there is a loss of sparseness in the feature vectors. Many works on LS-SVM are focused on improving support vectors representation in the least squares approach, since they correspond to the only vectors that must be stored for further usage of the machine, which can also be directly used as a reduced subset that represents the initial one. The proposed classifier incorporates the advantages of either SVM and LS-SVM: automatic detection of support vectors and a solution obtained simply by the solution of systems of linear equations. IP-LSSVM was compared with other sparse LS-SVM classifiers from literature, and RRS+LS-SVM. The experiments were performed on four important benchmark databases in Machine Learning and on two artificial databases created to show visually the support vectors detected. The results show that IP-LSSVM represents a viable alternative to SVMs, since both have similar features, supported by literature results and yet IP-LSSVM has a simpler and more understandable formulation.     

基于向量投影的支撑向量预选取

支撑向量机是近年来新兴的模式识别方法,在解决小样本、非线性及高维模式识别问题中表现出了突出的优点．但在支撑向量机中,支撑向量的选取相当困难,这也成为限制其应用的瓶颈问题．该文对支撑向量机的机理经过认真分析,研究其支撑向量的分布特性,在不影响分类性能的前提下,提出了基于向量投影的支撑向量预选取法,从训练样本中预先选择具有一定特征的边界向量来代替训练样本进行训练,这样就减少了训练样本,大大加快了支撑向量机的训练速度。     

基于LSI和SVM的文本分类研究

文本分类技术是文本数据挖掘的基础和核心，是基于自然语言处理技术和机器学习算法的一个具体应用。特征选择和分类算法是文本分类中两个最关键的技术，该文提出了利用潜在语义索引进行特征提取和降维，并结合支持向量机(SVM)算法进行多类分类，实验结果显示与向量空间模型(VSM)结合SVM方法和LSI结合K近邻(KNN)方法相比，取得了更好的效果，在文本类别数较少、类别划分比较清晰的情况下可以达到实用效果。     

New Least Squares Support Vector Machines Based on Matrix Patterns

Support vector machine (SVM), as an effective method in classification problems, tries to find the optimal hyperplane that maximizes the margin between two classes and can be obtained by solving a constrained optimization criterion using quadratic programming (QP). This QP leads to higher computational cost. Least squares support vector machine (LS-SVM), as a variant of SVM, tries to avoid the above shortcoming and obtain an analytical solution directly from solving a set of linear equations instead of QP. Both SVM and LS-SVM operate directly on patterns represented by vector, i.e., before applying SVM or LS-SVM to a pattern, any non-vector pattern such as an image has to be first vectorized into a vector pattern by some techniques like concatenation. However, some implicit structural or local contextual information may be lost in this transformation. Moreover, as the dimension d of the weight vector in SVM or LS-SVM with the linear kernel is equal to the dimension d ₁ × d ₂ of the original input pattern, as a result, the higher the dimension of a vector pattern is, the more space is needed for storing it. In this paper, inspired by the method of feature extraction directly based on matrix patterns and the advantages of LS-SVM, we propose a new classifier design method based on matrix patterns, called MatLSSVM, such that the new method can not only directly operate on original matrix patterns, but also efficiently reduce memory for the weight vector (d) from d ₁ × d ₂ to d ₁ + d ₂. However like LS-SVM, MatLSSVM inherits LS-SVM’s existence of unclassifiable regions when extended to multi-class problems. Thus with the fuzzy version of LS-SVM, a corresponding fuzzy version of MatLSSVM (MatFLSSVM) is further proposed to remove unclassifiable regions effectively for multi-class problems. Experimental results on some benchmark datasets show that the proposed method is competitive in classification performance compared to LS-SVM, fuzzy LS-SVM (FLS-SVM), more-recent MatPCA and MatFLDA. In addition, more importantly, the idea used here has a possibility of providing a novel way of constructing learning model.     

Color image segmentation using automatic pixel classification with support vector machine

Automatic segmentation of images is a very challenging fundamental task in computer vision and one of the most crucial steps toward image understanding. In this paper, we present a color image segmentation using automatic pixel classification with support vector machine (SVM). First, the pixel-level color feature is extracted in consideration of human visual sensitivity for color pattern variations, and the image pixel's texture feature is represented via steerable filter. Both the pixel-level color feature and texture feature are used as input of SVM model (classifier). Then, the SVM model (classifier) is trained by using fuzzy c-means clustering (FCM) with the extracted pixel-level features. Finally, the color image is segmented with the trained SVM model (classifier). This image segmentation not only can fully take advantage of the local information of color image, but also the ability of SVM classifier. Experimental evidence shows that the proposed method has a very effective segmentation results and computational behavior, and decreases the time and increases the quality of color image segmentation in compare with the state-of-the-art segmentation methods recently proposed in the literature.     

基于SVM和ICA的视频帧字幕自动定位与提取

视频字幕蕴涵了丰富语义,可以用来对相应视频流进行高级语义标注,但由于先前视频字幕提取考虑的只是如何尽可能定义好字幕特征,而忽视了分类学习机自身的学习推广能力.针对这一局限性,提出了一种基于支持向量机和独立分量分析的视频帧字幕定位与提取算法.该算法是首先将原始图象帧分割成N×N大小子块,同时将每个子块标注为字幕块和非字幕块两类;然后从每个子块提取能够保持相互高阶独立的独立分量特征去训练支持向量机分类器;最后结合金字塔模型和去噪方法,用训练好的支持向量机来实现对视频字幕区域自动定位提取.由于支持向量机能够在样本不是很多的情况下,具有良好的分类推广能力以及能使独立成分特征之间彼此保持高阶独立性,与其他视频帧字幕定位提取算法比较的结果表明,该算法具有明显的优点.     

Color image segmentation using pixel wise support vector machine classification

Image segmentation is an important tool in image processing and can serve as an efficient front end to sophisticated algorithms and thereby simplify subsequent processing. In this paper, we present a color image segmentation using pixel wise support vector machine (SVM) classification. Firstly, the pixel-level color feature and texture feature of the image, which is used as input of SVM model (classifier), are extracted via the local homogeneity model and Gabor filter. Then, the SVM model (classifier) is trained by using FCM with the extracted pixel-level features. Finally, the color image is segmented with the trained SVM model (classifier). This image segmentation not only can fully take advantage of the local information of color image, but also the ability of SVM classifier. Experimental evidence shows that the proposed method has a very effective segmentation results and computational behavior, and decreases the time and increases the quality of color image segmentation in comparison with the state-of-the-art segmentation methods recently proposed in the literature.     

An improved protein fold recognition with support vector machines

Predicting the three‐dimensional structure (fold) of a protein is a key problem in molecular biology. It is also interesting issue for statistical methods recognition. In this paper a multi‐class support vector machine (SVM) classifier is used on a real world data set. The SVM is a binary classifier, but protein fold recognition is a multi‐class problem. So several new approaches to deal with this issue are presented including a modification of the well‐known one‐versus‐one strategy. However, in this strategy the number of different binary classifiers that must be trained is quickly increasing with the number of classes. The methods proposed in this paper show how this problem can be overcome.     

Training SVM email classifiers using very large imbalanced dataset

The Internet has been flooded with spam emails, and during the last decade there has been an increasing demand for reliable anti-spam email filters. The problem of filtering emails can be considered as a classification problem in the field of supervised learning. Theoretically, many mature technologies, for example, support vector machines (SVM), can be used to solve this problem. However, in real enterprise applications, the training data are typically collected via honeypots and thus are always of huge amounts and highly biased towards spam emails. This challenges both efficiency and effectiveness of conventional technologies. In this article, we propose an undersampling method to compress and balance the training set used for the conventional SVM classifier with minimal information loss. The key observation is that we can make a trade-off between training set size and information loss by carefully defining a similarity measure between data samples. Our experiments show that the SVM classifier provides a better performance by applying our compressing and balancing approach.     

Least Square Transduction Support Vector Machine

Support vector machine (SVM) is a general and powerful learning machine, which adopts supervised manner. However, for many practical machine learning and data mining applications, unlabeled training examples are readily available but labeled ones are very expensive to be obtained. Therefore, semi-supervised learning emerges as the times require. At present, the combination of SVM and semi-supervised learning principle such as transductive learning has attracted more and more attentions. Transductive support vector machine (TSVM) learns a large margin hyperplane classifier using labeled training data, but simultaneously force this hyperplane to be far away from the unlabeled data. TSVM might seem to be the perfect semi-supervised algorithm since it combines the powerful regularization of SVMs and a direct implementation of the clustering assumption, nevertheless its objective function is non-convex and then it is difficult to be optimized. This paper aims to solve this difficult problem. We apply least square support vector machine to implement TSVM, which can ensure that the objective function is convex and the optimization solution can then be easily found by solving a set of linear equations. Simulation results demonstrate that the proposed method can exploit unlabeled data to yield good performance effectively.     

模糊粗糙集在污水处理过程中的应用

污水处理过程是个典型的多变量、非线性、具有强外部干扰的复杂工业过程。因此,如何进行污水处理工艺过程的故障诊断成为一个研究课题。本文结合基于模糊粗糙集的属性选择方法及支持向量机分类机理．提出一种新的故障诊断方法。首先使用基于粗糙集的属性选择（FR-FS）对过程特征变量进行约简,去除数据中的噪声,并降低过程数据的维数,获得具有代表性的过程特征信息．同时充分利用SVM的良好推广性能．提高了预测分类精度。最后将改故障诊断方法应用于广州沥浯污水处理厂．仿真结果表明了该方法的优越性。     

非结构化道路区域检测的协同学习方法

非结构化道路区域检测是智能车环境感知的重要问题。提出基于多方向Gabor纹理直方图的SVM分类器,并将其与直方图反向投影器组合,建立了协同学习框架。在实际运行中,两个学习器可以相互为对方提供标注样本进行更新,既提高了在线学习能力,又回避了自学习过程经常导致的模型漂移问题。经实验测试,协同学习机制显著改善了道路检测性能。     

基于最优ABC-SVM算法的P2P流量识别

目前对等网络（Peer-to-Peer,P2P）流量的识别是网络管理研究的热门话题。基于支持向量机（Support Vector Machine , SVM）的P2P流量识别方法是常用的P2P流量识别方法之一。然而SVM的性能主要受参数和其使用特征的影响,而传统的方法则是将SVM的参数优化和特征选择问题分开处理,因此这样很难获得整体性能最优的SVM分类器。本论文提出了一种基于最优人工蜂群算法和支持向量机相结合的P2P流量识别方法,利用人工蜂群算法,将SVM的参数和特征选择问题视为最优化问题同步处理,可以获得整体性能最优的参数和特征子集。在真实的P2P数据上的实验结果表明提出的方法具有很好的自适应性和分类精度,能够同时获取特征子集和SVM参数的最优解,提高SVM分类器的整体性能。     

p范数正则化支持向量机分类算法

L2范数罚支持向量机(Support vector machine,SVM)是目前使用最广泛的分类器算法之一,同时实现特征选择和分类器构造的L1范数和L0范数罚SVM算法也已经提出.但是,这两个方法中,正则化阶次都是事先给定,预设p=2或p=1.而我们的实验研究显示,对于不同的数据,使用不同的正则化阶次,可以改进分类算法的预测准确率.本文提出p范数正则化SVM分类器算法设计新模式,正则化范数的阶次p可取范围为02范数罚SVM,L1范数罚SVM和L0范数罚SVM.     

超声乳腺肿瘤的全自动SVM检测与水平集分割算法

超声图像检测是当前乳腺癌诊断的主要辅助手段之一.为实现超声乳腺肿瘤的计算机自动辅助诊断,提出一种基于支持向量机(SVM)目标检测与水平集图像分割相结合的全自动肿瘤提取算法.首先提取超声图像训练集的分块特征来训练SVM分类器,对测试集图像进行检测得到可疑病灶区域;然后提取可疑区域边缘作为水平集的初始轮廓,使用加入Bhattacharyya距离项的Chan-Vese主动轮廓改进模型进行可疑病灶区域的轮廓演化,得到准确的轮廓;最后综合面积、位置、灰度、纹理等因素设计区域评价筛选准则,去除可疑病灶中的干扰区域,得到最终的肿瘤分割结果.在真实病例数据集上的测试结果表明,利用该算法在良恶性肿瘤检测分割中均有较好表现.     

An efficient augmented Lagrangian method for support vector machine

ABSTRACT

Support vector machine (SVM) has proved to be a successful approach for machine learning. Two typical SVM models are the L1-loss model for support vector classification (SVC) and ε-L1-loss model for support vector regression (SVR). Due to the non-smoothness of the L1-loss function in the two models, most of the traditional approaches focus on solving the dual problem. In this paper, we propose an augmented Lagrangian method for the L1-loss model, which is designed to solve the primal problem. By tackling the non-smooth term in the model with Moreau–Yosida regularization and the proximal operator, the subproblem in augmented Lagrangian method reduces to a non-smooth linear system, which can be solved via the quadratically convergent semismooth Newton's method. Moreover, the high computational cost in semismooth Newton's method can be significantly reduced by exploring the sparse structure in the generalized Jacobian. Numerical results on various datasets in LIBLINEAR show that the proposed method is competitive with the most popular solvers in both speed and accuracy.     

Optimal feature selection for support vector machines

Selecting relevant features for support vector machine (SVM) classifiers is important for a variety of reasons such as generalization performance, computational efficiency, and feature interpretability. Traditional SVM approaches to feature selection typically extract features and learn SVM parameters independently. Independently performing these two steps might result in a loss of information related to the classification process. This paper proposes a convex energy-based framework to jointly perform feature selection and SVM parameter learning for linear and non-linear kernels. Experiments on various databases show significant reduction of features used while maintaining classification performance.