一种基于拟牛顿法的大类别分类算法

支持向量机利用接近边界的少数向量来构造一个最优分类面。然而当两类中的样本数量差别悬殊时，PSVM算法则会过度拟合样本量大的那一类，而对样本量很小的那一类的错分率相当高。为解决此问题，本文提出了一种改进的支持向量机算于拟牛顿法的大类别分类算法。同时，这个问题也是大类别分类问题所采用的留一法面临的问题，在DFP-PSVM的基础上，提出了基于拟牛顿法的大类别分类算法。通过仿真实验证实了此算法在精度上优于PSVM算法。     

一种基于近似支撑矢量机(PSVM)的交通目标分类方法

本文介绍了支撑向量机的特点,给出了实际应用中传统支撑矢量机存在的问题。为了克服支撑矢量机算法的不足,引入了一种近似支撑矢量机（PSVM）算法,并将此算法用于交通目标的分类识别。实验结果表明此算法比BP神经网络法准确率高,比传统的SVM法的效率高。     

模糊临近支持向量机

临近支持向量机(ProximalSupportVectorMachine)是近年提出的一种新的支持向量机。尽管在处理模式分类问题中速度远远超过传统支持向量机,临近支持向量机理论没有考虑不同输入样本点可能会对最优分类超平面的形成产生不同影响。文章给不同的训练样本赋予不同的模糊隶属度,以此来改进临近支持向量机。实验证明这种改进后的模糊临近支持向量机拥有一些临近支持向量机所没有的性能。     

基于近似支持向量机的Web文本分类研究

文本分类技术是知识管理系统实现知识有效组织、存储和检索的重要手段.本文提出了一种新的基于近似支持向量机的分类算法,并将该分类算法应用于文本分类分析.实验过程中与现有的分类方法比较,新的分类方法具有训练速度快、分类精度比较高的优点.     

Implementation of machine-learning classification in remote sensing: an applied review

Machine learning offers the potential for effective and efficient classification of remotely sensed imagery. The strengths of machine learning include the capacity to handle data of high dimensionality and to map classes with very complex characteristics. Nevertheless, implementing a machine-learning classification is not straightforward, and the literature provides conflicting advice regarding many key issues. This article therefore provides an overview of machine learning from an applied perspective. We focus on the relatively mature methods of support vector machines, single decision trees (DTs), Random Forests, boosted DTs, artificial neural networks, and k-nearest neighbours (k-NN). Issues considered include the choice of algorithm, training data requirements, user-defined parameter selection and optimization, feature space impacts and reduction, and computational costs. We illustrate these issues through applying machine-learning classification to two publically available remotely sensed data sets.     

ON IMAGE CLASSIFICATION: CITY IMAGES VS. LANDSCAPES

Grouping images into semantically meaningful categories using low-level visual features is a challenging and important problem in content-based image retrieval. Based on these groupings, effective indices can be built for an image database. In this paper, we show how a specific high-level classification problem (city images vs landscapes) can be solved from relatively simple low-level features geared for the particular classes. We have developed a procedure to qualitatively measure the saliency of a feature towards a classification problem based on the plot of the intra-class and inter-class distance distributions. We use this approach to determine the discriminative power of the following features: color histogram, color coherence vector, DCT coefficient, edge direction histogram, and edge direction coherence vector. We determine that the edge direction-based features have the most discriminative power for the classification problem of interest here. A weighted k-NN classifier is used for the classification which results in an accuracy of 93.9% when evaluated on an image database of 2716 images using the leave-one-out method. This approach has been extended to further classify 528 landscape images into forests, mountains, and sunset/sunrise classes. First, the input images are classified as sunset/sunrise images vs forest & mountain images (94.5% accuracy) and then the forest & mountain images are classified as forest images or mountain images (91.7% accuracy). We are currently identifying further semantic classes to assign to images as well as extracting low level features which are salient for these classes. Our final goal is to combine multiple 2-class classifiers into a single hierarchical classifier.     

医学影像库的一种智能分类方法

针对医学影像库信息量大、关联信息多、对象复杂的特点,将粗糙集算法与一种近似的支撑矢量机算法相结合实现了对医学影像库的正常、异常分类.粗糙集算法有效地降低了医学影像库的维度,而非线性的近似支撑矢量机算法则克服了标准支撑矢量机在实际应用中表现出来的算法速度慢、算法过于复杂而难于实现以及检测阶段运算量大等缺陷.实践证明了该方法的确具备简单、快速、高效的特点.     

Iterative support vector machine with guaranteed accuracy and run time

Abstract: Using a conjugate gradient method, a novel iterative support vector machine (FISVM) is proposed, which is capable of generating a new non‐linear classifier. We attempt to solve a modified primal problem of proximal support vector machine (PSVM) and show that the solution of the modified primal problem reduces to solving just a system of linear equations as opposed to a quadratic programming problem in SVM. This algorithm not only has no requirement for special optimization solvers, such as linear or quadratic programming tools, but also guarantees fast convergence. The full algorithm merely needs four lines of MATLAB codes, which gives results that are similar to or better than that of several new learning algorithms, in terms of classification accuracy. Besides, the proposed stand‐alone approach is capable of dealing with instability of classification performance of smooth support vector machine, generalized proximal support vector machine, PSVM and reduced support vector machine. Experiments carried out on UCI datasets show the effectiveness of our approach.     

Performance analysis of support vector machines classifiers in breast cancer mammography recognition

Support vector machine (SVM) is a supervised machine learning approach that was recognized as a statistical learning apotheosis for the small-sample database. SVM has shown its excellent learning and generalization ability and has been extensively employed in many areas. This paper presents a performance analysis of six types of SVMs for the diagnosis of the classical Wisconsin breast cancer problem from a statistical point of view. The classification performance of standard SVM (St-SVM) is analyzed and compared with those of the other modified classifiers such as proximal support vector machine (PSVM) classifiers, Lagrangian support vector machines (LSVM), finite Newton method for Lagrangian support vector machine (NSVM), Linear programming support vector machines (LPSVM), and smooth support vector machine (SSVM). The experimental results reveal that these SVM classifiers achieve very fast, simple, and efficient breast cancer diagnosis. The training results indicated that LSVM has the lowest accuracy of 95.6107 %, while St-SVM performed better than other methods for all performance indices (accuracy = 97.71 %) and is closely followed by LPSVM (accuracy = 97.3282). However, in the validation phase, the overall accuracies of LPSVM achieved 97.1429 %, which was superior to LSVM (95.4286 %), SSVM (96.5714 %), PSVM (96 %), NSVM (96.5714 %), and St-SVM (94.86 %). Value of ROC and MCC for LPSVM achieved 0.9938 and 0.9369, respectively, which outperformed other classifiers. The results strongly suggest that LPSVM can aid in the diagnosis of breast cancer.     

An all-pair quantum SVM approach for big data multiclass classification

In this paper, we discuss a quantum approach for the all-pair multiclass classification problem. In an all-pair approach, there is one binary classification problem for each pair of classes, and so there are k(k???1)/2 classifiers for a k-class classification problem. As compared to the classical multiclass support vector machine that can be implemented with polynomial run time complexity, our approach exhibits exponential speedup due to quantum computing. The quantum all-pair algorithm can also be used with other classification algorithms, and a speedup gain can be achieved as compared to their classical counterparts.     

分片支撑矢量机

文中借鉴了分段线性识别的基本思想,提出了分片支撑矢量机模型.该模型首先将特征空间剖分成若干子空间,在每个子空间中基于支撑矢量机构造一个最优分类面,然后,将各个分类面链接起来构成一个分片最优分类面以逼近理论上的最优分类超曲面.同时,文中还从理论上分析探讨了其推广能力的界,为分片支撑矢量机模型提供了坚实的基础.最后,经典双螺旋线数据实验结果表明,相对于传统支撑矢量机,分片支撑矢量机的计算速度、分类能力以及推广能力均有了明显提高.     

一种基于PSVM的多类分类方法

为克服传统支持向量机不能处理交叉数据分类问题,Mangasarian等人提出一种新的分类方法PSVM,该方法可有效解决交叉数据两分类问题,但用PSVM解决多分类问题还报道不多。为此,提出一种基于PSVM的多分类方法（M-PSVM）,并探讨训练样本比例与分类精度之间关系。在UCI数据集上的测试结果表明,M-PSVM与传统SVM分类性能相当,且当训练样本比例小时,效果更优;此外,在入侵检测数据集上的初步实验表明,M-PSVM可有效改进少数类的分类精度,因而为求解数据不平衡下的分类问题提供了新的思路,进一步的实验验证正在进行。     

Turning from TF-IDF to TF-IGM for term weighting in text classification

Massive textual data management and mining usually rely on automatic text classification technology. Term weighting is a basic problem in text classification and directly affects the classification accuracy. Since the traditional TF-IDF (term frequency & inverse document frequency) is not fully effective for text classification, various alternatives have been proposed by researchers. In this paper we make comparative studies on different term weighting schemes and propose a new term weighting scheme, TF-IGM (term frequency & inverse gravity moment), as well as its variants. TF-IGM incorporates a new statistical model to precisely measure the class distinguishing power of a term. Particularly, it makes full use of the fine-grained term distribution across different classes of text. The effectiveness of TF-IGM is validated by extensive experiments of text classification using SVM (support vector machine) and kNN (k nearest neighbors) classifiers on three commonly used corpora. The experimental results show that TF-IGM outperforms the famous TF-IDF and the state-of-the-art supervised term weighting schemes. In addition, some new findings different from previous studies are obtained and analyzed in depth in the paper.     

A self-adaptive synthetic over-sampling technique for imbalanced classification

Traditionally, in supervised machine learning, (a significant) part of the available data (usually 50%-80%) is used for training and the rest—for validation. In many problems, however, the data are highly imbalanced in regard to different classes or does not have good coverage of the feasible data space which, in turn, creates problems in validation and usage phase. In this paper, we propose a technique for synthesizing feasible and likely data to help balance the classes as well as to boost the performance in terms of confusion matrix as well as overall. The idea, in a nutshell, is to synthesize data samples in close vicinity to the actual data samples specifically for the less represented (minority) classes. This has also implications to the so-called fairness of machine learning. In this paper, we propose a specific method for synthesizing data in a way to balance the classes and boost the performance, especially of the minority classes. It is generic and can be applied to different base algorithms, for example, support vector machines, k-nearest neighbour classifiers deep neural, rule-based classifiers, decision trees, and so forth. The results demonstrated that (a) a significantly more balanced (and fair) classification results can be achieved and (b) that the overall performance as well as the performance per class measured by confusion matrix can be boosted. In addition, this approach can be very valuable for the cases when the number of actual available labelled data is small which itself is one of the problems of the contemporary machine learning.     

增量密度加权近似支持向量机

近似支持向量机((PSVM)是一个正则化最小二乘问题,有解析解,但是它失去了支持向量机(SVM)的稀疏 性,使得所有的训练样例都成为支持向量。为了有效地控制近似支持向量机的稀疏性,提出了增量密度加权近似支持 向量机(mWPSVM),它在训练集中选取最基本的支持向量。实验表明,IvWPSVM方法与SVM, PSVM和DWPS- VM方法相比,其精度相似,收敛速度快,可有效地控制近似支持向量机的稀疏性。     

Statistical Properties and Adaptive Tuning of Support Vector Machines

In this paper we consider the statistical aspects of support vector machines (SVMs) in the classification context, and describe an approach to adaptively tuning the smoothing parameter(s) in the SVMs. The relation between the Bayes rule of classification and the SVMs is discussed, shedding light on why the SVMs work well. This relation also reveals that the misclassification rate of the SVMs is closely related to the generalized comparative Kullback-Leibler distance (GCKL) proposed in Wahba (1999, Scholkopf, Burges, & Smola (Eds.), Advances in Kernel Methods—Support Vector Learning. Cambridge, MA: MIT Press). The adaptive tuning is based on the generalized approximate cross validation (GACV), which is an easily computable proxy of the GCKL. The results are generalized to the unbalanced case where the fraction of members of the classes in the training set is different than that in the general population, and the costs of misclassification for the two kinds of errors are different. The main results in this paper have been obtained in several places elsewhere. Here we take the opportunity to organize them in one place and note how they fit together and reinforce one another. Mostly the work of the authors is reviewed.     

支持向量机多类目标分类器的结构简化研究

由于支持向量机(SVM)在模式识别和回归分析中有着独特优势，因此成为近来研究的热点，其优势主要体现在处理非线性和高维数据问题方面。最初的SVM特别适合解决两类目标分类问题，而对于多类目标分类，则需将其转化为多个两类目标分类问题，相应地即可构造多个两类目标子分类器，但由于这种情况导致了分类器结构的过于复杂，从而导致判决速度的降低。为了快速地进行分类．提出了一种简化结构的多类目标分类器，其不仅使得子分类器数目大大减少，而且使分类速度明显提高；同时对其分类精度和复杂度进行了对比分析。实验结果证明。该分类器是有效的。     

Extreme learning machine for regression and multiclass classification

Due to the simplicity of their implementations, least square support vector machine (LS-SVM) and proximal support vector machine (PSVM) have been widely used in binary classification applications. The conventional LS-SVM and PSVM cannot be used in regression and multiclass classification applications directly, although variants of LS-SVM and PSVM have been proposed to handle such cases. This paper shows that both LS-SVM and PSVM can be simplified further and a unified learning framework of LS-SVM, PSVM, and other regularization algorithms referred to extreme learning machine (ELM) can be built. ELM works for the "generalized" single-hidden-layer feedforward networks (SLFNs), but the hidden layer (or called feature mapping) in ELM need not be tuned. Such SLFNs include but are not limited to SVM, polynomial network, and the conventional feedforward neural networks. This paper shows the following: 1) ELM provides a unified learning platform with a widespread type of feature mappings and can be applied in regression and multiclass classification applications directly; 2) from the optimization method point of view, ELM has milder optimization constraints compared to LS-SVM and PSVM; 3) in theory, compared to ELM, LS-SVM and PSVM achieve suboptimal solutions and require higher computational complexity; and 4) in theory, ELM can approximate any target continuous function and classify any disjoint regions. As verified by the simulation results, ELM tends to have better scalability and achieve similar (for regression and binary class cases) or much better (for multiclass cases) generalization performance at much faster learning speed (up to thousands times) than traditional SVM and LS-SVM.