Evaluating the Generalization Ability of Support Vector Machines through the Bootstrap

The well-known bounds on the generalizationability of learning machines, based on the Vapnik–Chernovenkis (VC) dimension,are very loose when applied to Support Vector Machines (SVMs).In this work we evaluate the validity of the assumption that these bounds are,nevertheless, good indicators of the generalization ability of SVMs.We show that this assumption is, in general, true and assessits correctness, in a statistical sense, on several pattern recognition benchmarks throughthe use of the bootstrap technique.     

一种用于文本分类的语义SVM及其在线学习算法

该文利用SVM在小训练样本集条件下仍有高泛化能力的特性,结合文本分类问题中同类别文本的特征在特征空间中具有聚类性分布的特点,提出一种使用语义中心集代替原训练样本集作为训练样本和支持向量的SVM:语义SVM。文中给出语义中心集的生成步骤,进而给出语义SVM的在线学习(在线分类知识积累)算法框架,以及基于SMO算法的在线学习算法的实现。实验结果说明语义SVM及其在线学习算法具有巨大的应用潜力:不仅在线学习速度和分类速度相对于标准SVM及其简单增量算法有数量级提高,而且分类准确率方面具有一定优势。     

Full-body person recognition system

We describe a system that learns from examples to recognize persons in images taken indoors. Images of full-body persons are represented by color-based and shape-based features. Recognition is carried out through combinations of Support Vector Machine (SVM) classifiers. Different types of multi-class strategies based on SVMs are explored and compared to k-Nearest Neighbors classifiers. The experimental results show high recognition rates and indicate the strength of SVM-based classifiers to improve both generalization and run-time performance. The system works in real-time.     

Support Vector Machines for TCP traffic classification

Support Vector Machines (SVM) represent one of the most promising Machine Learning (ML) tools that can be applied to the problem of traffic classification in IP networks. In the case of SVMs, there are still open questions that need to be addressed before they can be generally applied to traffic classifiers. Having being designed essentially as techniques for binary classification, their generalization to multi-class problems is still under research. Furthermore, their performance is highly susceptible to the correct optimization of their working parameters. In this paper we describe an approach to traffic classification based on SVM. We apply one of the approaches to solving multi-class problems with SVMs to the task of statistical traffic classification, and describe a simple optimization algorithm that allows the classifier to perform correctly with as little training as a few hundred samples. The accuracy of the proposed classifier is then evaluated over three sets of traffic traces, coming from different topological points in the Internet. Although the results are relatively preliminary, they confirm that SVM-based classifiers can be very effective at discriminating traffic generated by different applications, even with reduced training set sizes.     

An online incremental learning support vector machine for large-scale data

Support Vector Machines (SVMs) have gained outstanding generalization in many fields. However, standard SVM and most of modified SVMs are in essence batch learning, which make them unable to handle incremental learning or online learning well. Also, such SVMs are not able to handle large-scale data effectively because they are costly in terms of memory and computing consumption. In some situations, plenty of Support Vectors (SVs) are produced, which generally means a long testing time. In this paper, we propose an online incremental learning SVM for large data sets. The proposed method mainly consists of two components: the learning prototypes (LPs) and the learning Support Vectors (LSVs). LPs learn the prototypes and continuously adjust prototypes to the data concept. LSVs are to get a new SVM by combining learned prototypes with trained SVs. The proposed method has been compared with other popular SVM algorithms and experimental results demonstrate that the proposed algorithm is effective for incremental learning problems and large-scale problems.     

A support vector machine with a hybrid kernel and minimal Vapnik-Chervonenkis dimension

We present a mechanism to train support vector machines (SVMs) with a hybrid kernel and minimal Vapnik-Chervonenkis (VC) dimension. After describing the VC dimension of sets of separating hyperplanes in a high-dimensional feature space produced by a mapping related to kernels from the input space, we proposed an optimization criterion to design SVMs by minimizing the upper bound of the VC dimension. This method realizes a structural risk minimization and utilizes a flexible kernel function such that a superior generalization over test data can be obtained. In order to obtain a flexible kernel function, we develop a hybrid kernel function and a sufficient condition to be an admissible Mercer kernel based on common Mercer kernels (polynomial, radial basis function, two-layer neural network, etc.). The nonnegative combination coefficients and parameters of the hybrid kernel are determined subject to the minimal upper bound of the VC dimension of the learning machine. The use of the hybrid kernel results in a better performance than those with a single common kernel. Experimental results are discussed to illustrate the proposed method and show that the SVM with the hybrid kernel outperforms that with a single common kernel in terms of generalization power.     

支持向量机多类分类算法新研究

支持向量机最初是针对两类分类问题提出的,如何将其推广至多类分类问题是当前SVM研究中的热点问题之一。主要针对支持向量机多类分类方法中的分解重构法进行了深入分析,详细讨论了影响分类器性能的两个关键因素：分解策略和组合策略,并通过实验验证了该观点。最后,通过实验对比了包括M-ary 支持向量机和模糊支持向量机的SVM多类分类方法。     

一种基于支持向量机思想的无监督聚类算法

支持向量机作为一种新的机器学习方法,由于其建立在结构风险最小化准则之上,而不是仅仅使经验风险达到最小,从而使对支持向量分类器具有较好的推广能力。本文分析了支持向量机在解决无监督分类问题上的不足,提出一种基于支持向量机思想的最大间距的聚类新方法。实验结果表明,该算法能成功地解决很多非监督分类问题。     

一种支持向量机集成的核选择方法

核选择问题是支持向量机（Support Vector Machine,SVM）建模中的一个关键问题,虽然支持向量机具有良好的泛化性能,但其性能受核函数的影响比较明显,而对于一个给定问题,选择合适的核函数及参数通常很困难。提出一种基于SVM集成的核选择方法,利用不同的核函数构造子SVM学习器,然后对子学习器的预测结果集成。提出的核选择方法将SVM集成学习与核选择同时进行,不仅避免了单个SVM的核选择对泛化能力的影响,而且可以获得良好的泛化能力。在UCI标准数据集上的结果说明了提出的方法的有效性。     

基于支持向量机的遥感图像舰船目标识别方法

针对高分辨率遥感图像舰船目标识别问题,提出了一种基于支持向量机的舰船目标分类方法。支持向量机(SVM)是一类新型机器学习方法,基于结构风险最小化归纳原则,具有出色的学习能力。与传统的方法相比,支持向量机不但结构简单,而且技术性能特别是泛化能力明显提高。该文简要介绍了有关统计学习理论和支持向量机算法,将支持向量机应用于遥感图像舰船目标识别,并同传统的舰船识别方法进行了相关的对比实验,实验结果说明本文提出的分类器在识别性能上明显优于其它传统分类器,具有更高的识别性能率。