一种改进的多项式核支持向量机文本分类器 *

多项式核函数由于具有良好的泛化性能而受到重视,并被研究用于文本分类问题。针对多项式核学习 能力较差的缺点,将学习能力较强的条件正定核与多项式核构成一个混合核函数作为改进的多项式核。实验表 明,改进的多项式核SVM文本分类器的分类效果要好于多项式核SVM文本分类器。     

基于Gabor小波和支持向量机的人脸识别算法中若干问题研究

归纳出了基于Gabor小波和支持向量机（SVM）的人脸识别算法在实际应用中所遇到的三个问题，即主元分析（PCA）降维过程中累积贡献率的选择，多项式核函数阶数的选择以及决策函数的确定。给出了累积贡献率和多项式核函数的阶数选择规则，提出了支持向量机和最大值（MAX）相结合的分类决策方法。最后，利用AT＆T人脸库进行仿真比较研究。     

基于遗传算法的加权支持向量机的短期电力负荷预测

建立在统计学习理论的VC维理论和结构风险最小化原理基础之上的支持向量机（SVM）在理论上保证了模型的最大泛化能力,因此将支持向量机理论应用于电力负荷预测可以获得很好的效果,但是传统SVM回归预测算法对于不同的样本均采用相同的参数,无法体现各样本的重要程度的区别,而且将支持向量机理论应用于实际中也存在对样本数据进行特征选择和对支持向量机模型参数进行选择的问题。因此提出了将遗传算法应用于短期的电力负荷预测中来对加权支持向量机模型进行特征选择和模型参数确定。     

组合核函数SVM在特定领域文本分类中的应用

面向特定领域文本分类的实际应用,存在大量样本相互掺杂的现象,使其无法线性表述,在SVM中引入核函数可以有效地解决非线性分类的问题,而选择不同的核函数可以构造不同的SVM,其识别性能也不同,因此,选择合适的核函数及其参数优化成为SVM的关键.本文基于单核核函数的性质,对多项式核函数与径向基核函数进行线性加权,构建具有良好的泛化能力与良好的学习能力的组合核函数.仿真实验结果表明,在选择正确参数的情况下,组合核函数SVM的宏平均准确率、宏平均召回率及宏平均综合分类率都明显优于线性核、多项式核与径向基核,而且能够兼顾准确率与召回率.     

基于SVM的中文组块分析

基于SVM(support vector machine)理论的分类算法,由于其完善的理论基础和良好的实验结果,目前已逐渐引起国内外研究者的关注。和其他分类算法相比,基于结构风险最小化原则的SVM在小样本模式识别中表现较好的泛化能力。文本组块分析作为句法分析的预处理阶段,通过将文本划分成一组互不重叠的片断,来达到降低句法分析的难度。本文将中文组块识别问题看成分类问题,并利用SVM加以解决。实验结果证明,SVM算法在汉语组块识别方面是有效的,在哈尔滨工业大学树库语料测试的结果是F=88.67%,并且特别适用于有限的汉语带标信息的情况。     

基于W^1 2再生核支持向量机的模式分类研究

支持向量机是基于统计学习理论的模式分类器。它通过结构风险最小化准则和核函数方法，较好地解决了模式分类器复杂性和推广性之间的矛盾，引起了大家对模式识别领域的极大关注。近年来，支持向量机在手写体识别、人脸识别、文本分类等领域取得了很大的成功。文章将一种新的核函数用于虹膜识别，并与传统的多项式核函数、高斯核函数进行了比较。初步结果显示了该核函数的应用潜力。     

基于PAC-Bayes边界理论的SVM模型选择方法

PAC-Bayes边界理论融合了贝叶斯定理和随机分类器的结构风险最小化原理,它作为一个理论框架,能有效评价机器学习算法的泛化性能。针对支持向量机（SVM）模型选择问题,通过分析PAC-Bayes边界理论框架及其在SVM上的应用,将PAC-Bayes边界理论与基于交叉验证的网格搜索法相结合,提出一种基于PAC-Bayes边界的SVM模型选择方法（PBB-GS）,实现快速优选SVM的惩罚系数和核函数参数。UCI数据集的实验结果表明该方法优选出的参数能使SVM具有较高的泛化性能,并具有简便快速、参数选择准确的优点,能有效改善SVM模型选择问题。     

基于W_2~1再生核支持向量机的模式分类研究

支持向量机是基于统计学习理论的模式分类器。它通过结构风险最小化准则和核函数方法,较好地解决了模式分类器复杂性和推广性之间的矛盾,引起了大家对模式识别领域的极大关注。近年来,支持向量机在手写体识别、人脸识别、文本分类等领域取得了很大的成功。文章将一种新的核函数用于虹膜识别,并与传统的多项式核函数、高斯核函数进行了比较。初步结果显示了该核函数的应用潜力。     

基于W12再生核支持向量机的模式分类研究

支持向量机是基于统计学习理论的模式分类器.它通过结构风险最小化准则和核函数方法,较好地解决了模式分类器复杂性和推广性之间的矛盾,引起了大家对模式识别领域的极大关注.近年来,支持向量机在手写体识别、人脸识别、文本分类等领域取得了很大的成功.文章将一种新的核函数用于虹膜识别,并与传统的多项式核函数、高斯核函数进行了比较.初步结果显示了该核函数的应用潜力.     

支持向量机分类器遥感图像分类研究

SVM分类器核函数的选择以及参数的设置直接影响系统的泛化能力和运行速度。引入交叉验证技术和栅格搜索技术,对径向基核、多项式核和Sigmoid核函数应用于图像多类别分类的性能进行理论推导、测试及分析,求得三种核函数应用于SVM分类器的性能,并证明了栅格搜索寻找最优参数的有效性。最后通过对TM 6波段BSQ格式遥感图像进行分类对比证明了SVM分类器核函数用于TM图像分类的可行性及高效性。     

A Tutorial on Support Vector Machines for Pattern Recognition

The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and non-separable data, working through a non-trivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light.     

基于SVM的特征筛选方法及其若干应用

对于拟合问题，传统的模式识别特征筛选方法以各特征量对训练数据拟合能力的贡献为取舍标准，未考虑经验风险最小化和结构风险最小化间的差别，不能获得预报能力最强的特征筛选结果。为此我们提出了结合支持向量回归法与留一法的特征筛选新算法，并将它试用于镍氢电池材料和氧化铝溶出率两套实验数据集的特征筛选。     

A two-stage feature selection method for text categorization by using information gain,principal component analysis and genetic algorithm

Text categorization is widely used when organizing documents in a digital form. Due to the increasing number of documents in digital form, automated text categorization has become more promising in the last ten years. A major problem of text categorization is its large number of features. Most of those are irrelevant noise that can mislead the classifier. Therefore, feature selection is often used in text categorization to reduce the dimensionality of the feature space and to improve performance. In this study, two-stage feature selection and feature extraction is used to improve the performance of text categorization. In the first stage, each term within the document is ranked depending on their importance for classification using the information gain (IG) method. In the second stage, genetic algorithm (GA) and principal component analysis (PCA) feature selection and feature extraction methods are applied separately to the terms which are ranked in decreasing order of importance, and a dimension reduction is carried out. Thereby, during text categorization, terms of less importance are ignored, and feature selection and extraction methods are applied to the terms of highest importance; thus, the computational time and complexity of categorization is reduced. To evaluate the effectiveness of dimension reduction methods on our purposed model, experiments are conducted using the k-nearest neighbour (KNN) and C4.5 decision tree algorithm on Reuters-21,578 and Classic3 datasets collection for text categorization. The experimental results show that the proposed model is able to achieve high categorization effectiveness as measured by precision, recall and F-measure.     

结合优化的文档频和PA的特征选择方法*

特征空间的高维特点限制了分类算法的选择,影响了分类器的设计和准确度,降低了分类器的泛化能力,从而出现分类器过拟合的现象,因此需要进行特征选择以避免维数灾难。首先简单分析了几种经典特征选择方法,总结了它们的不足;然后给出了一个优化的文档频方法,并用它过滤掉一些词条以降低文本矩阵的稀疏性;最后应用模式聚合(PA)理论建立文本集的向量空间模型,从分类贡献的角度强化词条的作用,消减原词条矩阵中包含的冗余模式,从而有效地降低了向量空间的维数,提高了文本分类的精度和速度。实验结果表明此种综合性特征选择方法效果良好。     

A support vector machine with a hybrid kernel and minimal Vapnik-Chervonenkis dimension

We present a mechanism to train support vector machines (SVMs) with a hybrid kernel and minimal Vapnik-Chervonenkis (VC) dimension. After describing the VC dimension of sets of separating hyperplanes in a high-dimensional feature space produced by a mapping related to kernels from the input space, we proposed an optimization criterion to design SVMs by minimizing the upper bound of the VC dimension. This method realizes a structural risk minimization and utilizes a flexible kernel function such that a superior generalization over test data can be obtained. In order to obtain a flexible kernel function, we develop a hybrid kernel function and a sufficient condition to be an admissible Mercer kernel based on common Mercer kernels (polynomial, radial basis function, two-layer neural network, etc.). The nonnegative combination coefficients and parameters of the hybrid kernel are determined subject to the minimal upper bound of the VC dimension of the learning machine. The use of the hybrid kernel results in a better performance than those with a single common kernel. Experimental results are discussed to illustrate the proposed method and show that the SVM with the hybrid kernel outperforms that with a single common kernel in terms of generalization power.     

RQEA-SVR在交通流预测中的应用

建立在统计学习理论和结构风险最小化准则基础上的支持向量回归（SVR）是处理小样本数据回归问题的有利工具,SVR的参数选取直接影响其学习性能和泛化能力。文中将SVR参数选取看作是参数的组合优化问题,确定组合优化问题的目标函数,采用实数量子进化算法（RQEA）求解组合优化问题进而优选SVR参数,形成RQEA-SVR,并应用RQEA-SVR求解交通流预测问题。仿真试验表明RQEA是优选SVR参数的有效方法,解决交通流预测问题具有优良的性能。     

基于独立性理论的文本分类特征选择方法

特征与各个文档类在文本集中的独立程度体现了特征的代表性,文本分类的特征选择过程是选择能够提高分类性能的高代表性特征的过程。基于该原理提出DHChi2和EIBA 2种新的文本分类特征选择方法,对这2种方法进行合理的组合。实验结果表明,独立性理论应用于文本分类特征选择有利于提高分类性能。     

Boosting视角

AdaBoost是Boosting家族中的最基础的代表算法。本文主要介绍了AdaBoost的泛化错误分析及其与结构风险最小化和VC维、支持向量机及margin理论的关系,并从游戏理论和统计学视点分别对AdaBoost进行了理解和解释,以期提供Boosting的一个较为全面的视角。     

Model selection for the LS-SVM. Application to handwriting recognition

The support vector machine (SVM) is a powerful classifier which has been used successfully in many pattern recognition problems. It has also been shown to perform well in the handwriting recognition field. The least squares SVM (LS-SVM), like the SVM, is based on the margin-maximization principle performing structural risk minimization. However, it is easier to train than the SVM, as it requires only the solution to a convex linear problem, and not a quadratic problem as in the SVM. In this paper, we propose to conduct model selection for the LS-SVM using an empirical error criterion. Experiments on handwritten character recognition show the usefulness of this classifier and demonstrate that model selection improves the generalization performance of the LS-SVM.