A new maximal-margin spherical-structured multi-class support vector machine

Support vector machines (SVMs), initially proposed for two-class classification problems, have been very successful in pattern recognition problems. For multi-class classification problems, the standard hyperplane-based SVMs are made by constructing and combining several maximal-margin hyperplanes, and each class of data is confined into a certain area constructed by those hyperplanes. Instead of using hyperplanes, hyperspheres that tightly enclosed the data of each class can be used. Since the class-specific hyperspheres are constructed for each class separately, the spherical-structured SVMs can be used to deal with the multi-class classification problem easily. In addition, the center and radius of the class-specific hypersphere characterize the distribution of examples from that class, and may be useful for dealing with imbalance problems. In this paper, we incorporate the concept of maximal margin into the spherical-structured SVMs. Besides, the proposed approach has the advantage of using a new parameter on controlling the number of support vectors. Experimental results show that the proposed method performs well on both artificial and benchmark datasets.     

基于支持向量机的多分类增量学习算法

支持向量机被成功地应用在分类和回归问题中，但是由于其需要求解二次规划，使得支持向量机在求解大规模数据上具有一定的缺陷，尤其是对于多分类问题，现有的支持向量机算法具有太高的算法复杂性。该文提出一种基于支持向量机的增量学习算法，适合多分类问题，并将之用于解决实际问题。     

A Self-Organizing TS-Type Fuzzy Network With Support Vector Learning and its Application to Classification Problems

A self-organizing Takagi-Sugeno (TS)-type fuzzy network with support vector learning (SOTFN-SV) is proposed in this paper. The proposed SOTFN-SV is inspired by analysis of TS-type fuzzy systems and composite-kernel support vector machine (SVM). SOTFN-SV is a fuzzy system constructed by the hybridization of fuzzy clustering and SVM. The antecedent part of SOTFN-SV is generated via fuzzy clustering of the input data, and then SVM is used to tune the consequent part parameters to give the network better generalization performance. For demonstration, SOTFN-SV is applied to several classification problems, especially the skin color classification problem. In the skin color classification application, each color pixel is represented by hue and saturation (HS) color space. To represent color information by histogram as accurately as possible, a nonuniform partition of HS space is proposed. For comparison, SVMs and other fuzzy systems trained by SVM or neural networks are applied to the same classification problems. The advantages of SOTFN-SV are verified by comparisons with the results of these methods.     

A corporate credit rating model using multi-class support vector machines with an ordinal pairwise partitioning approach

Predicting corporate credit-rating using statistical and artificial intelligence (AI) techniques has received considerable research attention in the literature. In recent years, multi-class support vector machines (MSVMs) have become a very appealing machine-learning approach due to their good performance. Until now, researchers have proposed a variety of techniques for adapting support vector machines (SVMs) to multi-class classification, since SVMs were originally devised for binary classification. However, most of them have only focused on classifying samples into nominal categories; thus, the unique characteristic of credit-rating - ordinality - seldom has been considered in the proposed approaches. This study proposes a new type of MSVM classifier (named OMSVM) that is designed to extend the binary SVMs by applying an ordinal pairwise partitioning (OPP) strategy. Our model can efficiently and effectively handle multiple ordinal classes. To validate OMSVM, we applied it to a real-world case of bond rating. We compared the results of our model with those of conventional MSVM approaches and other AI techniques including MDA, MLOGIT, CBR, and ANNs. The results showed that our proposed model improves the performance of classification in comparison to other typical multi-class classification techniques and uses fewer computational resources.     

一种新的模糊补偿多类支持向量机

支持向量机是Vapnik等学者在统计学习理论的基础上提出的一种新的机器学习方法。针对支持向量机理论中的多类分类问题和对于噪音数据的敏感性，本文提出了一种模糊补偿多类支持向量机算法FC-SVM。该算法是在Weston等人提出的多类SVM分类器的直接构造方法中引入模糊补偿函数，针对每个输入数据对分类结果的两方面影响，将目标函数中的惩罚项不仅进行了模糊化，而且对于分类情况进行了加权补偿，并重构了优化问题及其约束条件，然后重构了Lagrange公式，给出了理论推导。在充分的数值实验基础上，将文中提出的方法应用于建设银行个人房贷的信用评估系统中，得到了较好的实验结果。     

GENERATING FUZZY RULES FROM TRAINING DATA CONTAINING NOISE FOR HANDLING CLASSIFICATION PROBLEMS

It is obvious that one of the important tasks in a fuzzy system is to find a set of rules to deal with a specific classification problem. In recent years, many researchers focused on the research topic of generating fuzzy rules from training data for handling classification problems. In a previous paper, we presented an algorithm to construct membership functions and to generate fuzzy rules from training examples. In this paper, we extend that work to propose a new algorithm to generate fuzzy rules from training data containing noise to deal with classification problems. The proposed algorithm gets a higher classification accuracy rate and generates fewer fuzzy rules and fewer input attributes in the antecedent portions of the generated fuzzy rules.     

Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems

Artificial neural networks often achieve high classification accuracy rates, but they are considered as black boxes due to their lack of explanation capability. This paper proposes the new rule extraction algorithm RxREN to overcome this drawback. In pedagogical approach the proposed algorithm extracts the rules from trained neural networks for datasets with mixed mode attributes. The algorithm relies on reverse engineering technique to prune the insignificant input neurons and to discover the technological principles of each significant input neuron of neural network in classification. The novelty of this algorithm lies in the simplicity of the extracted rules and conditions in rule are involving both discrete and continuous mode of attributes. Experimentation using six different real datasets namely iris, wbc, hepatitis, pid, ionosphere and creditg show that the proposed algorithm is quite efficient in extracting smallest set of rules with high classification accuracy than those generated by other neural network rule extraction methods.     

Support Vector Machines for TCP traffic classification

Support Vector Machines (SVM) represent one of the most promising Machine Learning (ML) tools that can be applied to the problem of traffic classification in IP networks. In the case of SVMs, there are still open questions that need to be addressed before they can be generally applied to traffic classifiers. Having being designed essentially as techniques for binary classification, their generalization to multi-class problems is still under research. Furthermore, their performance is highly susceptible to the correct optimization of their working parameters. In this paper we describe an approach to traffic classification based on SVM. We apply one of the approaches to solving multi-class problems with SVMs to the task of statistical traffic classification, and describe a simple optimization algorithm that allows the classifier to perform correctly with as little training as a few hundred samples. The accuracy of the proposed classifier is then evaluated over three sets of traffic traces, coming from different topological points in the Internet. Although the results are relatively preliminary, they confirm that SVM-based classifiers can be very effective at discriminating traffic generated by different applications, even with reduced training set sizes.     

一种新的多类支持向量机决策方法及其应用

讨论和比较了现有的几种多类SVM方法．在此基础上,提出了一种组合多个两类分类器结果的多类SVM决策方法．在该方法中,定义了新的决策函数,其值是在传统投票决策值的基础上乘以不同分类器的权重．新的多类SVM在一定程度上解决了传统投票决策方法的不可分区域问题,因此具有更好的分类性能．最后,将新方法作为关键技术应用于故障诊断实例,实际诊断结果证明了所提多类SVM决策方法的优越性．     

An efficient orientation distance–based discriminative feature extraction method for multi-classification

Feature extraction is an important step before actual learning. Although many feature extraction methods have been proposed for clustering, classification and regression, very limited work has been done on multi-class classification problems. This paper proposes a novel feature extraction method, called orientation distance–based discriminative (ODD) feature extraction, particularly designed for multi-class classification problems. Our proposed method works in two steps. In the first step, we extend the Fisher Discriminant idea to determine an appropriate kernel function and map the input data with all classes into a feature space where the classes of the data are well separated. In the second step, we put forward two variants of ODD features, i.e., one-vs-all-based ODD and one-vs-one-based ODD features. We first construct hyper-plane (SVM) based on one-vs-all scheme or one-vs-one scheme in the feature space; we then extract one-vs-all-based or one-vs-one-based ODD features between a sample and each hyper-plane. These newly extracted ODD features are treated as the representative features and are thereafter used in the subsequent classification phase. Extensive experiments have been conducted to investigate the performance of one-vs-all-based and one-vs-one-based ODD features for multi-class classification. The statistical results show that the classification accuracy based on ODD features outperforms that of the state-of-the-art feature extraction methods.     

Fingerprint classification using one-vs-all support vector machines dynamically ordered with naı¨ve Bayes classifiers

Fingerprint classification reduces the number of possible matches in automated fingerprint identification systems by categorizing fingerprints into predefined classes. Support vector machines (SVMs) are widely used in pattern classification and have produced high accuracy when performing fingerprint classification. In order to effectively apply SVMs to multi-class fingerprint classification systems, we propose a novel method in which the SVMs are generated with the one-vs-all (OVA) scheme and dynamically ordered with na?¨ve Bayes classifiers. This is necessary to break the ties that frequently occur when working with multi-class classification systems that use OVA SVMs. More specifically, it uses representative fingerprint features as the FingerCode, singularities and pseudo ridges to train the OVA SVMs and na?¨ve Bayes classifiers. The proposed method has been validated on the NIST-4 database and produced a classification accuracy of 90.8% for five-class classification with the statistical significance. The results show the benefits of integrating different fingerprint features as well as the usefulness of the proposed method in multi-class fingerprint classification.     

Relaxed constraints support vector machines for noisy data

Real-world data collected for computer-based applications are frequently impure. Differentiation of outliers and noisy data from normal ones is a major task in data mining applications. On the other hand, elimination of noisy and outlier data from training samples of a dataset may lead to over-fitting or information loss. A fuzzy support vector machine (FSVM) provides an effective means to deal with this problem. It reduces the effect of the noisy data and outliers by using a fuzzy membership functions. In this paper, a new formation for SVMs is introduced that considers importance degrees for training samples. The constraints of the SVM are converted to fuzzy inequalities. The proposed method, RSVM, shows better efficiency in the classification of data in different domains. Especially, using the proposed RSVM for multi-class classification of arrhythmia disease is presented at the end of this paper as a practical case study to show the effectiveness of the proposed system.     

Support vector machines for face recognition

Support vector machines (SVMs) have been recently proposed as a new learning network for bipartite pattern recognition. In this paper, SVMs incorporated with a binary tree recognition strategy are proposed to tackle the multi-class face recognition problem. The binary tree extends naturally, the pairwise discrimination capability of the SVMs to the multi-class scenario. Two face databases are used to evaluate the proposed method. The performance of the SVMs based face recognition is compared with the standard eigenface approach, and also the more recently proposed algorithm called the nearest feature line (NFL).     

Extraction of rules from artificial neural networks for nonlinearregression

Neural networks (NNs) have been successfully applied to solve a variety of application problems including classification and function approximation. They are especially useful as function approximators because they do not require prior knowledge of the input data distribution and they have been shown to be universal approximators. In many applications, it is desirable to extract knowledge that can explain how Me problems are solved by the networks. Most existing approaches have focused on extracting symbolic rules for classification. Few methods have been devised to extract rules from trained NNs for regression. This article presents an approach for extracting rules from trained NNs for regression. Each rule in the extracted rule set corresponds to a subregion of the input space and a linear function involving the relevant input attributes of the data approximates the network output for all data samples in this subregion. Extensive experimental results on 32 benchmark data sets demonstrate the effectiveness of the proposed approach in generating accurate regression rules.     

Design of hierarchical fuzzy model for classification problem using GAs

This paper proposes a new hierarchical fuzzy model (HFM) to solve the classification problem. The developed classification model comprises of two stages; one is to generate the fuzzy IF–THEN rules for each subsystem and the other is to determine the classification unit. For the classification problem, number of rules and the correct classification rate are the fundamental requirements. In this paper, we also advance two genetic algorithms (GAs) to tune the HFM. One is used to determine the combination of the input features for each subsystem on the HFM and the other is to reduce the number of rules in each fuzzy subsystem. The performance has been tested by simulations on the well known Wine and Iris databases. Simulations demonstrate that the proposed HFM under a few rules can provide sufficiently high classification rate even with higher feature dimensions.     

Data mining with a simulated annealing based fuzzy classification system

In this paper, the use of simulated annealing (SA) metaheuristic for constructing a fuzzy classification system is presented. In several previous investigations, the capability of fuzzy systems to solve different kinds of problems has been demonstrated. Simulated annealing based fuzzy classification system (SAFCS), hybridizes the learning capability of SA metaheuristic with the approximate reasoning method of fuzzy systems. The objective of this paper is to illustrate the ability of SA to develop an accurate fuzzy classifier. The use of SA in classification is an attempt to effectively explore and exploit the large search space usually associated with classification problems, and find the optimum set of fuzzy if–then rules. The SAFCS would be capable to extract accurate fuzzy classification rules from input data sets, and applies them to classify new data instances in different predefined groups or classes. Experiments are performed with eight UCI data sets. The results indicate that the proposed SAFCS achieves competitive results in comparison with several well-known classification algorithms.     

基于概率投票策略的多类支持向量机及应用

传统的支持向量机是基于两类问题提出的,如何将其有效地推广至多类分类仍是一个研究的热点问题。在分析比较现有支持向量机多类分类OVO方法存在的问题及缺点的基础上,该文提出一种新的基于概率投票策略的多类分类方法。在该策略中,充分考虑了OVO方法中各个两类支持向量机分类器的差异,并将该差异反映到投票分值上。所提多类支持向量机方法不仅具有较好的分类性能,而且有效解决了传统投票策略中存在的拒分区域问题。将基于概率投票的多分类支持向量机作为关键技术应用于实际齿轮箱故障诊断,并与传统投票策略的结果进行对比,表明所提方法的上述优点。     

A NEW METHOD TO GENERATE FUZZY RULES FROM TRAINING INSTANCES FOR HANDLING CLASSIFICATION PROBLEMS

A major task in developing a fuzzy classification system is to generate a set of fuzzy rules from training instances to deal with a specific classification problem. In recent years, many methods have been developed to generate fuzzy rules from training instances. We present a new method to generate fuzzy rules from training instances to deal with the Iris data classification problem. The proposed method can discard some useless input attributes to improve the average classification accuracy rate. It can obtain a higher average classification accuracy rate and it generates fewer fuzzy rules and fewer input fuzzy sets in the generated fuzzy rules than the existing methods.     

Using Gaussian process based kernel classifiers for credit rating forecasting

The subprime mortgage crisis have triggered a significant economic decline over the world. Credit rating forecasting has been a critical issue in the global banking systems. The study trained a Gaussian process based multi-class classifier (GPC), a highly flexible probabilistic kernel machine, using variational Bayesian methods. GPC provides full predictive distributions and model selection simultaneously. During training process, the input features are automatically weighted by their relevances with respect to the output labels. Benefiting from the inherent feature scaling scheme, GPCs outperformed convectional multi-class classifiers and support vector machines (SVMs). In the second stage, conventional SVMs enhanced by feature selection and dimensionality reduction schemes were also compared with GPCs. Empirical results indicated that GPCs still performed the best.