Improving the SVM gender classification accuracy using clustering and incremental learning

Gender recognition has been playing a very important role in various applications such as human–computer interaction, surveillance, and security. Nonlinear support vector machines (SVMs) were investigated for the identification of gender using the Face Recognition Technology (FERET) image face database. It was shown that SVM classifiers outperform the traditional pattern classifiers (linear, quadratic, Fisher linear discriminant, and nearest neighbour). In this context, this paper aims to improve the SVM classification accuracy in the gender classification system and propose new models for a better performance. We have evaluated different SVM learning algorithms; the SVM‐radial basis function with a 5% outlier fraction outperformed other SVM classifiers. We have examined the effectiveness of different feature selection methods. AdaBoost performs better than the other feature selection methods in selecting the most discriminating features. We have proposed two classification methods that focus on training subsets of images among the training images. Method 1 combines the outcome of different classifiers based on different image subsets, whereas method 2 is based on clustering the training data and building a classifier for each cluster. Experimental results showed that both methods have increased the classification accuracy.     

Comparison of artificial neural networks and support vector machine classifiers for land cover classification in Northern China using a SPOT-5 HRG image

This article presents a sufficient comparison of two types of advanced non-parametric classifiers implemented in remote sensing for land cover classification. A SPOT-5 HRG image of Yanqing County, Beijing, China, was used, in which agriculture and forest dominate land use. Artificial neural networks (ANNs), including the adaptive backpropagation (ABP) algorithm, Levenberg–Marquardt (LM) algorithm, Quasi-Newton (QN) algorithm and radial basis function (RBF) were carefully tested. The LM–ANN and RBF–ANN, which outperform the other two, were selected to make a detailed comparison with support vector machines (SVMs). The experiments show that those well-trained ANNs and SVMs have no significant difference in classification accuracy, but the SVM usually performs slightly better. Analysis of the effect of the training set size highlights that the SVM classifier has great tolerance on a small training set and avoids the problem of insufficient training of ANN classifiers. The testing also illustrates that the ANNs and SVMs can vary greatly with regard to training time. The LM–ANN can converge very quickly but not in a stable manner. By contrast, the training of RBF–ANN and SVM classifiers is fast and can be repeatable.     

基于证据理论的多类分类支持向量机集成

针对多类分类问题,研究支持向量机集成中的分类器组合架构与方法.分析已有的多类级和两类级支持向量机集成架构的不足后,提出两层的集成架构.在此基础上,研究基于证据理论的支持向量机度量层输出信息融合方法,针对一对多与一对一两种多类扩展策略,分别定义基本概率分配函数,并根据证据冲突程度采用不同的证据组合规则.在一对多策略下,采用经典的Dempster规则;在一对一策略下则提出一条新的规则,以组合冲突严重的证据.实验表明,两层架构优于多类级架构,证据理论方法能有效地利用两类支持向量机的度量层输出信息,取得了满意的结果.     

An extreme learning machine approach for speaker recognition

Over the last two decades, automatic speaker recognition has been an interesting and challenging problem to speech researchers. It can be classified into two different categories, speaker identification and speaker verification. In this paper, a new classifier, extreme learning machine, is examined on the text-independent speaker verification task and compared with SVM classifier. Extreme learning machine (ELM) classifiers have been proposed for generalized single hidden layer feedforward networks with a wide variety of hidden nodes. They are extremely fast in learning and perform well on many artificial and real regression and classification applications. The database used to evaluate the ELM and SVM classifiers is ELSDSR corpus, and the Mel-frequency Cepstral Coefficients were extracted and used as the input to the classifiers. Empirical studies have shown that ELM classifiers and its variants could perform better than SVM classifiers on the dataset provided with less training time.     

SVM在多源遥感图像分类中的应用研究

在利用遥感图像进行土地利用/覆盖分类过程中,可采用以下两种途径来提高分类精度:一是通过增加有利于分类的数据源,引入地理辅助数据和归一化植被指数(NDVI)来进行多源信息融合;二是选择更好的分类方法,例如支持向量机(SVM)学习方法,由于该方法克服了最大似然法和神经网络的弱点,非常适合高维、复杂的小样本多源数据的分类。为了提高多源遥感图像分类的精度,还研究了支持向量机在遥感图像分类中模型的选择,包括多类模型和核函数的选择。分类结果表明,支持向量机比传统的分类方法具有更高的精度,尤其是基于径向基核函数和一对一多类方法的支持向量机模型更适合多源遥感图像分类,因此,基于支持向量机的多源土地利用/覆盖分类能大大提高分类精度。     

Dynamic classification for video stream using support vector machine

A dynamic classification using the support vector machine (SVM) technique is presented in this paper as a new ‘incremental’ framework for multiple-classifying video stream data. The contribution of this study is the derivation of a unique, fast and simple to implement technique that allows multi-classification of behavioral motions based on an adaptation of the least-square SVM (LS-SVM) formulation. This dynamic approach leads to an extension of SVM beyond its current static image-based learning capabilities. The proposed incremental multi-classification method is applied to video stream data, which consists of an articulated humanoid model monitored by a surveillance camera. The initial supervised off-line learning phase is followed by a visual behavior data acquisition and then an incremental learning phase. The resulting error rate and the confidence level for the proposed technique demonstrate its validity and merits in articulated motion learning. Furthermore, the enabled online learning allows an adaptive domain knowledge insertion and provides the advantage of reducing both the model training time and the information storage requirements of the overall system which are both essential for dynamic soft computing applications.     

Benchmarking Least Squares Support Vector Machine Classifiers

In Support Vector Machines (SVMs), the solution of the classification problem is characterized by a (convex) quadratic programming (QP) problem. In a modified version of SVMs, called Least Squares SVM classifiers (LS-SVMs), a least squares cost function is proposed so as to obtain a linear set of equations in the dual space. While the SVM classifier has a large margin interpretation, the LS-SVM formulation is related in this paper to a ridge regression approach for classification with binary targets and to Fisher's linear discriminant analysis in the feature space. Multiclass categorization problems are represented by a set of binary classifiers using different output coding schemes. While regularization is used to control the effective number of parameters of the LS-SVM classifier, the sparseness property of SVMs is lost due to the choice of the 2-norm. Sparseness can be imposed in a second stage by gradually pruning the support value spectrum and optimizing the hyperparameters during the sparse approximation procedure. In this paper, twenty public domain benchmark datasets are used to evaluate the test set performance of LS-SVM classifiers with linear, polynomial and radial basis function (RBF) kernels. Both the SVM and LS-SVM classifier with RBF kernel in combination with standard cross-validation procedures for hyperparameter selection achieve comparable test set performances. These SVM and LS-SVM performances are consistently very good when compared to a variety of methods described in the literature including decision tree based algorithms, statistical algorithms and instance based learning methods. We show on ten UCI datasets that the LS-SVM sparse approximation procedure can be successfully applied.     

A telemedicine tool to detect pulmonary pathology using computerized pulmonary acoustic signal analysis

BackgroundDetection and monitoring of respiratory related illness is an important aspect in pulmonary medicine. Acoustic signals extracted from the human body are considered in detection of respiratory pathology accurately.ObjectivesThe aim of this study is to develop a prototype telemedicine tool to detect respiratory pathology using computerized respiratory sound analysis.MethodsAround 120 subjects (40 normal, 40 continuous lung sounds (20 wheeze and 20 rhonchi)) and 40 discontinuous lung sounds (20 fine crackles and 20 coarse crackles) were included in this study. The respiratory sounds were segmented into respiratory cycles using fuzzy inference system and then S-transform was applied to these respiratory cycles. From the S-transform matrix, statistical features were extracted. The extracted features were statistically significant with p < 0.05. To classify the respiratory pathology KNN, SVM and ELM classifiers were implemented using the statistical features obtained from of the data.ResultsThe validation showed that the classification rate for training for ELM classifier with RBF kernel was high compared to the SVM and KNN classifiers. The time taken for training the classifier was also less in ELM compared to SVM and KNN classifiers. The overall mean classification rate for ELM classifier was 98.52%.ConclusionThe telemedicine software tool was developed using the ELM classifier. The telemedicine tool has performed extraordinary well in detecting the respiratory pathology and it is well validated.     

基于蚁群算法的SVM模型选择研究

为了提高SVM的分类器性能,提出使用蚁群算法来指导SVM模型参数的选择,并针对采用RBF作为核函数的SVM进行了实验。然后将该方法与基于GA的SVM模型选择方法进行了比较。实验证明采用蚁群算法具有一定的优势,它能在较短的时间内寻找到最优解,且最终得到的分类结果优于遗传算法。     

Transfer learning between crop types for semantic segmentation of crops versus weeds in precision agriculture

Agricultural robots rely on semantic segmentation for distinguishing between crops and weeds to perform selective treatments and increase yield and crop health while reducing the amount of chemicals used. Deep‐learning approaches have recently achieved both excellent classification performance and real‐time execution. However, these techniques also rely on a large amount of training data, requiring a substantial labeling effort, both of which are scarce in precision agriculture. Additional design efforts are required to achieve commercially viable performance levels under varying environmental conditions and crop growth stages. In this paper, we explore the role of knowledge transfer between deep‐learning‐based classifiers for different crop types, with the goal of reducing the retraining time and labeling efforts required for a new crop. We examine the classification performance on three datasets with different crop types and containing a variety of weeds and compare the performance and retraining efforts required when using data labeled at pixel level with partially labeled data obtained through a less time‐consuming procedure of annotating the segmentation output. We show that transfer learning between different crop types is possible and reduces training times for up to 80%. Furthermore, we show that even when the data used for retraining are imperfectly annotated, the classification performance is within 2% of that of networks trained with laboriously annotated pixel‐precision data.     

基于蚁群算法的SVM模型选择研究

为了提高SVM的分类器性能，提出使用蚁群算法来指导SVM模型参数的选择，并针对采用RBF作为核函数的SVM进行了实验。然后将该方法与基于GA的SVM模型选择方法进行了比较。实验证明采用蚁群算法具有一定的优势，它能在较短的时间内寻找到最优解，且最终得到的分类结果优于遗传算法。     

基于支持向量机和距离度量的纹理分类

针对图象纹理分类问题，提出了一种将支持向量机和距离度量相结合，以构成两级组合分类器的分类方法，用该方法分类时，先采用距离度量进行前级分类，然后根据图象的纹理统计特征，采用欧氏距离来度量图象之间的相似性，若符合条件，则给出分类结果，否则拒识，并转入后级分类器，而后级分类器则采用一种新的模式分类方法－支持向量机进行分类，该组合分类方法不仅充分利用了支持向量机识别率高和距离度量速度快的优点，并且还利用距离度量的结果去指导支持向量机的训练和测试，由纹理图象分类的实验表明，该算法具有较高的效率和识别精度，同时也对推动支持向量机这一新的模式分类方法的实际应用具有积极意义。     

Classification of multi‐spectral remote sensing data using a local transfer function classifier

This paper introduces a new neural network, called the local transfer function classifier (LTF‐C), for classification of multi‐spectral remote sensing data. The network structure of LTF‐C is similar to that of the radial basis function neural network (RBF), but LTF‐C utilizes an entirely different learning algorithm. In particular, the network structure of LTF‐C is not predetermined, but changes dynamically during the learning. Such a learning algorithm fits well to the classification problem, and guarantees that the size of the network is as large as is needed. The classification results show that LTF‐C evidently has a better classification accuracy than the six other classifiers in the experiment.     

Reducing SVM classification time using multiple mirror classifiers.

We propose an approach that uses mirror point pairs and a multiple classifier system to reduce the classification time of a support vector machine (SVM). Decisions made with multiple simple classifiers formed from mirror pairs are integrated to approximate the classification rule of a single SVM. A coarse-to-fine approach is developed for selecting a given number of member classifiers. A clustering method, derived from the similarities between classifiers, is used for a coarse selection. A greedy strategy is then used for fine selection of member classifiers. Selected member classifiers are further refined by finding a weighted combination with a perceptron. Experiment results show that our approach can successfully speed up SVM decisions while maintaining comparable classification accuracy.     

Brain tumor segmentation and classification in MRI using SVM and its variants: a survey

The process of separation of brain tumor from normal brain tissues is Brain tumor segmentation. Segmentation of tumor from the MR images is a very challenging task as brain tumors are of different shapes and sizes. There are multiple phases to achieve the segmentation and the phases are pre-processing, segmentation, feature extraction, feature reduction, and classification of the tumor into benign and malignant. In this paper, Otsu thresholding is used in segmentation phase, Discrete Wavelet Transform (DWT) in feature extraction phase, Principal Component Analysis (PCA) in feature reduction phase and Support Vector Machine (SVM), Least Squared-Support Vector Machine (LS-SVM), Proximal Support Vector Machine (PSVM) and Twin Support Vector Machine (TWSVM) in the classification phase. We have compared the performances of all these classifiers, where TWSVM outperformed all other classifiers with 100% accuracy.

     

A multi-one-class dynamic classifier for adaptive digitization of document streams

In this paper, we present a new dynamic classifier design based on a set of one-class independent SVM for image data stream categorization. Dynamic or continuous learning and classification has been recently investigated to deal with different situations, like online learning of fixed concepts, learning in non-stationary environments (concept drift) or learning from imbalanced data. Most of solutions are not able to deal at the same time with many of these specificities. Particularly, adding new concepts, merging or splitting concepts are most of the time considered as less important and are consequently less studied, whereas they present a high interest for stream-based document image classification. To deal with that kind of data, we explore a learning and classification scheme based on one-class SVM classifiers that we call mOC-iSVM (multi-one-class incremental SVM). Even if one-class classifiers are suffering from a lack of discriminative power, they have, as a counterpart, a lot of interesting properties coming from their independent modeling. The experiments presented in the paper show the theoretical feasibility on different benchmarks considering addition of new classes. Experiments also demonstrate that the mOC-iSVM model can be efficiently used for tasks dedicated to documents classification (by image quality and image content) in a context of streams, handling many typical scenarii for concepts extension, drift, split and merge.     

Multispectral landuse classification using neural networks and support vector machines: one or the other,or both?

Land use classification is an important part of many remote sensing applications. A lot of research has gone into the application of statistical and neural network classifiers to remote‐sensing images. This research involves the study and implementation of a new pattern recognition technique introduced within the framework of statistical learning theory called Support Vector Machines (SVMs), and its application to remote‐sensing image classification. Standard classifiers such as Artificial Neural Network (ANN) need a number of training samples that exponentially increase with the dimension of the input feature space. With a limited number of training samples, the classification rate thus decreases as the dimensionality increases. SVMs are independent of the dimensionality of feature space as the main idea behind this classification technique is to separate the classes with a surface that maximizes the margin between them, using boundary pixels to create the decision surface. Results from SVMs are compared with traditional Maximum Likelihood Classification (MLC) and an ANN classifier. The findings suggest that the ANN and SVM classifiers perform better than the traditional MLC. The SVM and the ANN show comparable results. However, accuracy is dependent on factors such as the number of hidden nodes (in the case of ANN) and kernel parameters (in the case of SVM). The training time taken by the SVM is several magnitudes less.     

Evolutionary strategies for hyperparameters of support vector machines based on multi-scale radial basis function kernels

Kernel functions are used in support vector machines (SVM) to compute inner product in a higher dimensional feature space. SVM classification performance depends on the chosen kernel. The radial basis function (RBF) kernel is a distance-based kernel that has been successfully applied in many tasks. This paper focuses on improving the accuracy of SVM by proposing a non-linear combination of multiple RBF kernels to obtain more flexible kernel functions. Multi-scale RBF kernels are weighted and combined. The proposed kernel allows better discrimination in the feature space. This new kernel is proved to be a Mercer’s kernel. Furthermore, evolutionary strategies (ESs) are used for adjusting the hyperparameters of SVM. Training accuracy, the bound of generalization error, and subset cross-validation on training accuracy are considered to be objective functions in the evolutionary process. The experimental results show that the accuracy of multi-scale RBF kernels is better than that of a single RBF kernel. Moreover, the subset cross-validation on training accuracy is more suitable and it yields the good results on benchmark datasets.     

A multi-class SVM classification system based on learning methods from indistinguishable chinese official documents

Support Vector Machines (SVM) has been developed for Chinese official document classification in One-against-All (OAA) multi-class scheme. Several data retrieving techniques including sentence segmentation, term weighting, and feature extraction are used in preprocess. We observe that most documents of which contents are indistinguishable make poor classification results. The traditional solution is to add misclassified documents to the training set in order to adjust classification rules. In this paper, indistinguishable documents are observed to be informative for strengthening prediction performance since their labels are predicted by the current model in low confidence. A general approach is proposed to utilize decision values in SVM to identify indistinguishable documents. Based on verified classification results and distinguishability of documents, four learning strategies that select certain documents to training sets are proposed to improve classification performance. Experiments report that indistinguishable documents are able to be identified in a high probability and are informative for learning strategies. Furthermore, LMID that adds both of misclassified documents and indistinguishable documents to training sets is the most effective learning strategy in SVM classification for large set of Chinese official documents in terms of computing efficiency and classification accuracy.