超球体多类支持向量机理论

目前的多类分类器大多是经二分类器组合而成的,存在训练速度较慢的问题,在分类类别多的时候,会遇到很大困难,超球体多类支持向量机将超球体单类支持向量机扩展到多类问题,由于每类样本只参与一个超球体支持向量机的训练.因此,这是一种直接多类分类器,训练效率明显提高.为了有效训练超球体多类支持向量机,利用SMO算法思想,提出了超球体支持向量机的快速训练算法.同时对超球体多类支持向量机的推广能力进行了理论上的估计.数值实验表明,在分类类别较多的情况,这种分类器的训练速度有很大提高,非常适合解决类别数较多的分类问题.超球体多类支持向量机为研究快速直接多类分类器提供了新的思路.     

Multiclass LS-SVMs: Moderated Outputs and Coding-Decoding Schemes

A common way of solving the multiclass categorization problem is to reformulate the problem into a set of binary classification problems. Discriminative binary classifiers like, e.g., Support Vector Machines (SVMs), directly optimize the decision boundary with respect to a certain cost function. In a pragmatic and computationally simple approach, Least Squares SVMs (LS-SVMs) are inferred by minimizing a related regression least squares cost function. The moderated outputs of the binary classifiers are obtained in a second step within the evidence framework. In this paper, Bayes' rule is repeatedly applied to infer the posterior multiclass probabilities, using the moderated outputs of the binary plug-in classifiers and the prior multiclass probabilities. This Bayesian decoding motivates the use of loss function based decoding instead of Hamming decoding. For SVMs and LS-SVMs with linear kernel, experimental evidence suggests the use of one-versus-one coding. With a Radial Basis Function kernel one-versus-one and error correcting output codes yield the best performances, but simpler codings may still yield satisfactory results. This revised version was published online in August 2006 with corrections to the Cover Date.     

Detecting image orientation based on low-level visual content

Accurately and automatically detecting image orientation is of great importance in intelligent image processing. In this paper, we present automatic image orientation detection algorithms based on both the luminance (structural) and chrominance (color) low-level content features. The statistical learning support vector machines (SVMs) are used in our approach as the classifiers. The different sources of the extracted image features, as well as the binary classification nature of SVM, require our system to be able to integrate the outputs from multiple classifiers. Both static combiner (averaging) and trainable combiner (also based on SVMs) are proposed and evaluated in this work. Furthermore, two rejection options (regular and re-enforced ambiguity rejections) are employed to improve orientation detection accuracy by sieving out images with low confidence values during the classification. Large amounts of experiments have been conducted on a database of more than 14,000 images to validate our approaches. Discussions and future directions for this work are also addressed at the end of the paper.     

基于SVM的图像分类研究

图像分类技术有着重要的应用前景,而且对于基于内容的图像检索的发展会有积极的推动作用。多类图像分类是图像分类中的难点,对基于SVM的多类图像分类方法进行了研究,提出在二类支持向量机的基础上构造多类分类器的方法,实验结果证明和传统方法相比,分类准确率有了较大的提高。     

Combining eigenvector methods and support vector machines for detecting variability of Doppler ultrasound signals

In this paper, the multiclass support vector machines (SVMs) with the error correcting output codes (ECOC) were presented for detecting variabilities of the multiclass Doppler ultrasound signals. The ophthalmic arterial (OA) Doppler signals were recorded from healthy subjects, subjects suffering from OA stenosis, subjects suffering from ocular Behcet disease. The internal carotid arterial (ICA) Doppler signals were recorded from healthy subjects, subjects suffering from ICA stenosis, subjects suffering from ICA occlusion. Methods of combining multiple classifiers with diverse features are viewed as a general problem in various application areas of pattern recognition. Because of the importance of making the right decision, better classification procedures for Doppler ultrasound signals are searched. Decision making was performed in two stages: feature extraction by eigenvector methods and classification using the SVMs trained on the extracted features. The research demonstrated that the multiclass SVMs trained on extracted features achieved high accuracy rates.     

Covariance-guided landmine detection and discrimination using ground-penetrating radar data

Ground penetrating Radar (GPR) can detect and deliver the response signal from any buried kind of object like plastic or metallic landmines, stones, and wood sticks. It delivers three kinds of data: Ascan, Bscan, and Cscan. However, it cannot discriminate between landmines and inoffensive objects or ‘clutter.’ One-class classification is an alternative to detect landmines, especially, as landmines features data are unbalanced. In this article, we investigate the effectiveness of the Covariance-guided One-Class Support Vector Machine (COSVM) to detect, discriminate, and locate landmines efficiently. In fact, compared to existing one-class classifiers, the COSVM has the advantage of emphasizing low variance directions. Moreover, we will compare the one-class classification to multiclass classification to tease out the advantage of the former over the latter as data are unbalanced. Our method consists of extracting Ascan GPR data. Extracted features are used as an input for COSVM to discriminate between landmines and clutter. We provide an extensive evaluation of our detection method compared to other methods based on relevant state of the art one-class and multiclass classifiers, on the well-known MACADAM database. Our experimental results show clearly the superiority of using COSVM in landmine detection and localization.     

Parallelizing multiclass support vector machines for scalable image annotation

Machine learning techniques have facilitated image retrieval by automatically classifying and annotating images with keywords. Among them, Support Vector Machines (SVMs) are used extensively due to their generalization properties. SVM was initially designed for binary classifications. However, most classification problems arising in domains such as image annotation usually involve more than two classes. Notably, SVM training is a computationally intensive process especially when the training dataset is large. This paper presents a resource aware parallel multiclass SVM algorithm (named RAMSMO) for large-scale image annotation which partitions the training dataset into smaller binary chunks and optimizes SVM training in parallel using a cluster of computers. A genetic algorithm-based load balancing scheme is designed to optimize the performance of RAMSMO in balancing the computation of multiclass data chunks in heterogeneous computing environments. RAMSMO is evaluated in both experimental and simulation environments, and the results show that it reduces the training time significantly while maintaining a high level of accuracy in classifications.     

Fingerprint classification using one-vs-all support vector machines dynamically ordered with naı¨ve Bayes classifiers

Fingerprint classification reduces the number of possible matches in automated fingerprint identification systems by categorizing fingerprints into predefined classes. Support vector machines (SVMs) are widely used in pattern classification and have produced high accuracy when performing fingerprint classification. In order to effectively apply SVMs to multi-class fingerprint classification systems, we propose a novel method in which the SVMs are generated with the one-vs-all (OVA) scheme and dynamically ordered with na?¨ve Bayes classifiers. This is necessary to break the ties that frequently occur when working with multi-class classification systems that use OVA SVMs. More specifically, it uses representative fingerprint features as the FingerCode, singularities and pseudo ridges to train the OVA SVMs and na?¨ve Bayes classifiers. The proposed method has been validated on the NIST-4 database and produced a classification accuracy of 90.8% for five-class classification with the statistical significance. The results show the benefits of integrating different fingerprint features as well as the usefulness of the proposed method in multi-class fingerprint classification.     

Incorporating multiple SVMs for automatic image annotation

In this paper, a novel automatic image annotation system is proposed, which integrates two sets of support vector machines (SVMs), namely the multiple instance learning (MIL)-based and global-feature-based SVMs, for annotation. The MIL-based bag features are obtained by applying MIL on the image blocks, where the enhanced diversity density (DD) algorithm and a faster searching algorithm are applied to improve the efficiency and accuracy. They are further input to a set of SVMs for finding the optimum hyperplanes to annotate training images. Similarly, global color and texture features, including color histogram and modified edge histogram, are fed into another set of SVMs for categorizing training images. Consequently, two sets of image features are constructed for each test image and are, respectively, sent to the two sets of SVMs, whose outputs are incorporated by an automatic weight estimation method to obtain the final annotation results. Our proposed annotation approach demonstrates a promising performance for an image database of 12 000 general-purpose images from COREL, as compared with some current peer systems in the literature.     

Image mining by spectral features: A case study of scenery image classification

Spectral features of images, such as Gabor filters and wavelet transform can be used for texture image classification. That is, a classifier is trained based on some labeled texture features as the training set to classify unlabeled texture features of images into some pre-defined classes. The aim of this paper is twofold. First, it investigates the classification performance of using Gabor filters, wavelet transform, and their combination respectively, as the texture feature representation of scenery images (such as mountain, castle, etc.). A k-nearest neighbor (k-NN) classifier and support vector machine (SVM) are also compared. Second, three k-NN classifiers and three SVMs are combined respectively, in which each of the combined three classifiers uses one of the above three texture feature representations respectively, to see whether combining multiple classifiers can outperform the single classifier in terms of scenery image classification. The result shows that a single SVM using Gabor filters provides the highest classification accuracy than the other two spectral features and the combined three k-NN classifiers and three SVMs.     

A Novel and Principled Multiclass Support Vector Machine

Support vector machines (SVMs) have been demonstrated very efficient for binary classification problems; however, computationally efficient and effective multiclass SVMs are still missing. Most existing multiclass SVM classifiers are constructed either by combining multiple binary SVM classifiers, which often perform moderately for some problems, or by converting multiclass problems into one single optimization problem, which is unfortunately computationally expensive. To address these issues, a novel and principled multiclass SVM based on geometric properties of hyperspheres, termed SVMGH, is proposed in this paper. Different from existing SVM‐based methods that seek a cutting hyperplane between two classes, SVMGH draws the discriminative information of each class by constructing a minimum hypersphere containing all class members, and then defines a label function based on the geometric properties of the minimum hyperspheres. We prove theoretically the geometric properties of the minimum hyperspheres to guarantee the validation of SVMGH. The computational efficiency is enhanced by a data reduction strategy as well as a fast training method. Experimental results demonstrate that the proposed SVMGH shows better performance and higher computational efficiency than the state of the art on multiclassification problems while maintaining comparable performance and efficiency on binary classification problems.     

Use of particle swarm optimization for machinery fault detection

A study is presented on the application of particle swarm optimization (PSO) combined with other computational intelligence (CI) techniques for bearing fault detection in machines. The performance of two CI based classifiers, namely, artificial neural networks (ANNs) and support vector machines (SVMs) are compared. The time domain vibration signals of a rotating machine with normal and defective bearings are processed for feature extraction. The extracted features from original and preprocessed signals are used as inputs to the classifiers for detection of machine condition. The classifier parameters, e.g., the number of nodes in the hidden layer for ANNs and the kernel parameters for SVMs are selected along with input features using PSO algorithms. The classifiers are trained with a subset of the experimental data for known machine conditions and are tested using the remaining set of data. The procedure is illustrated using the experimental vibration data of a rotating machine. The roles of the number of features, PSO parameters and CI classifiers on the detection success are investigated. Results are compared with other techniques such as genetic algorithm (GA) and principal component analysis (PCA). The PSO based approach gave a test classification success rate of 98.6–100% which were comparable with GA and much better than with PCA. The results show the effectiveness of the selected features and the classifiers in the detection of the machine condition.     

Outlier detection based on a dynamic ensemble model: Applied to process monitoring

This paper focuses on outlier detection and its application to process monitoring. The main contribution is that we propose a dynamic ensemble detection model, of which one-class classifiers are used as base learners. Developing a dynamic ensemble model for one-class classification is challenging due to the absence of labeled training samples. To this end, we propose a procedure that can generate pseudo outliers, prior to which we transform outputs of all base classifiers to the form of probability. Then we use a probabilistic model to evaluate competence of all base classifiers. Friedman test along with Nemenyi test are used together to construct a switching mechanism. This is used for determining whether one classifier should be nominated to make the decision or a fusion method should be applied instead. Extensive experiments are carried out on 20 data sets and an industrial application to verify the effectiveness of the proposed method.     

Artificial neural networks and support vector machines with genetic algorithm for bearing fault detection

A study is presented to compare the performance of bearing fault detection using two different classifiers, namely, artificial neural networks (ANNs) and support vector machines (SMVs). The time-domain vibration signals of a rotating machine with normal and defective bearings are processed for feature extraction. The extracted features from original and preprocessed signals are used as inputs to the classifiers for two-class (normal or fault) recognition. The classifier parameters, e.g., the number of nodes in the hidden layer in case of ANNs and the radial basis function kernel parameter (width) in case of SVMs along with the selection of input features are optimized using genetic algorithms. The classifiers are trained with a subset of the experimental data for known machine conditions and are tested using the remaining set of data. The procedure is illustrated using the experimental vibration data of a rotating machine. The roles of different vibration signals and signal preprocessing techniques are investigated. The results show the effectiveness of the features and the classifiers in detection of machine condition.     

High dimensional low sample size activity recognition using geometric classifiers

Research on high dimension, low sample size (HDLSS) data has revealed their neighborless nature. This paper addresses the classification of HDLSS image or video data for human activity recognition. Existing approaches often use off-the-shelf classifiers such as nearest neighbor techniques or support vector machines and tend to ignore the geometry of underlying feature distributions. Addressing this issue, we investigate different geometric classifiers and affirm the lack of neighborhoods within HDLSS data. As this undermines proximity based methods and may cause over-fitting for discriminant methods, we propose a QR factorization approach to Nearest Affine Hull (NAH) classification which remedies the HDLSS dilemma and noticeably reduces time and memory requirements of existing methods. We show that the resulting non-parametric models provide smooth decision surfaces and yield efficient and accurate solutions in multiclass HDLSS scenarios. On several action recognition benchmarks, the proposed NAH classifier outperforms other instance based methods and shows competitive or superior performance than SVMs. In addition, for online settings, the proposed NAH method is faster than online SVMs.     

一种改进的最小二乘支持向量机及其应用

为了克服传统支持向量机训练速度慢、计算资源需求大等缺点,本文应用最小二乘支持向量机算法来解决分类问题。同时,本文指出了决策导向循环图算法的缺陷,采用自适应导向循环图思想来实现多类问题的分类。为了提高样本的学习速度,本文还将序贯最小优化算法与最小二乘支持向量机相结合,最终形成了ADAGLSSVM算法。考虑到最小二
乘支持向量机算法失去了支持向量的稀疏性,本文对支持向量作了修剪。实验结果表明,修剪后,分类器的识别精度和识别速度都得到了提高。     

A MapReduce-based distributed SVM algorithm for automatic image annotation

Machine learning techniques have facilitated image retrieval by automatically classifying and annotating images with keywords. Among them Support Vector Machines (SVMs) have been used extensively due to their generalization properties. However, SVM training is notably a computationally intensive process especially when the training dataset is large. This paper presents MRSMO, a MapReduce based distributed SVM algorithm for automatic image annotation. The performance of the MRSMO algorithm is evaluated in an experimental environment. By partitioning the training dataset into smaller subsets and optimizing the partitioned subsets across a cluster of computers, the MRSMO algorithm reduces the training time significantly while maintaining a high level of accuracy in both binary and multiclass classifications.