Fuzzy c-means based support vector machines classifier for perfume recognition

Identification of more than three perfumes is very difficult for the human nose. It is also a problem to recognize patterns of perfume odor with an electronic nose that has multiple sensors. For this reason, a new hybrid classifier has been presented to identify type of perfume from a closely similar data set of 20 different odors of perfumes. The structure of this hybrid technique is the combination of unsupervised fuzzy clustering c-mean (FCM) and supervised support vector machine (SVM). On the other hand this proposed soft computing technique was compared with the other well-known learning algorithms. The results show that the proposed hybrid algorithm’s accuracy is 97.5% better than the others.     

Response modeling with support vector machines

Support Vector Machine (SVM) employs Structural Risk Minimization (SRM) principle to generalize better than conventional machine learning methods employing the traditional Empirical Risk Minimization (ERM) principle. When applying SVM to response modeling in direct marketing, however, one has to deal with the practical difficulties: large training data, class imbalance and scoring from binary SVM output. For the first difficulty, we propose a way to alleviate or solve it through a novel informative sampling. For the latter two difficulties, we provide guidelines within SVM framework so that one can readily use the paper as a quick reference for SVM response modeling: use of different costs for different classes and use of distance to decision boundary, respectively. This paper also provides various evaluation measures for response models in terms of accuracies, lift chart analysis, and computational efficiency.     

支持向量机和一类模糊推理系统的等效性及其应用

支持向量机（ＳＶＭ）和模糊推理系统（ＦＩＳ）分别源于统计学习理论（ＳＬＴ）和认知学两个不同的领域．在一定约束条件下,提出并证明了ＳＶＭ 和一类基于规则的ＦＩＳ的函数等效性定理．在此基础上,提出基于ＳＶＭ 学习过程的ＦＩＳ（ＭＢＦＩＳ）的设计方法．ＭＢＦＩＳ继承了ＳＶＭ 良好的泛化能力和对“维数灾难”的避免能力,也继承了基于规则的ＦＩＳ的显式推理能力．Ｂｅｎｃｈｍａｒｋ数据实验表明,ＭＢＦＩＳ具有良好的分类性能．     

基于F-SVMs的多模型建模方法

针对全局模型难以精确描述复杂工业过程的问题,提出一种基于模糊支持向量机(F-SVMs)的多模型(F-SVMs MM)建模方法。用模糊支持向量分类算法(F-SVC)对输入数据进行预处理,得到多模型模糊隶属度;用模糊支持回归算法(F-SVR)建立多模型(MM)估计器。应用该方法对pH中和滴定过程进行建模,仿真结果表明,F-SVMs MM跟踪性能好、泛化能力强,比USOCPN方法和标准支持向量机(SVMs)方法具有更好的性能和推广能力。     

Decision confidence-based multi-level support vector machines

Support vector machines (SVM) have been showing high accuracy of prediction in many applications. However, as any statistical learning algorithm, SVM's accuracy drops if some of the training points are contaminated by an unknown source of noise. The choice of clean training points is critical to avoid the overfitting problem which occurs generally when the model is excessively complex, which is reflected by a high accuracy over the training set and a low accuracy over the testing set (unseen points). In this paper we present a new multi-level SVM architecture that splits the training set into points that are labeled as ‘easily classifiable’ which do not cause an increase in the model complexity and ‘non-easily classifiable’ which are responsible for increasing the complexity. This method is used to create an SVM architecture that yields on average a higher accuracy than a traditional soft margin SVM trained with the same training set. The architecture is tested on the well known US postal handwritten digit recognition problem, the Wisconsin breast cancer dataset and on the agitation detection dataset. The results show an increase in the overall accuracy for the three datasets. Throughout this paper the word confidence is used to denote the confidence over the decision as commonly used in the literature.     

On-line fuzzy modeling via clustering and support vector machines

In this paper, we propose a novel approach to identify unknown nonlinear systems with fuzzy rules and support vector machines. Our approach consists of four steps which are on-line clustering, structure identification, parameter identification and local model combination. The collected data are firstly clustered into several groups through an on-line clustering technique, then structure identification is performed on each group using support vector machines such that the fuzzy rules are automatically generated with the support vectors. Time-varying learning rates are applied to update the membership functions of the fuzzy rules. The modeling errors are proven to be robustly stable with bounded uncertainties by a Lyapunov method and an input-to-state stability technique. Comparisons with other related works are made through a real application of crude oil blending process. The results demonstrate that our approach has good accuracy, and this method is suitable for on-line fuzzy modeling.     

Evolutionary wrapper approaches for training set selection as preprocessing mechanism for support vector machines: Experimental evaluation and support vector analysis

One of the most powerful, popular and accurate classification techniques is support vector machines (SVMs). In this work, we want to evaluate whether the accuracy of SVMs can be further improved using training set selection (TSS), where only a subset of training instances is used to build the SVM model. By contrast to existing approaches, we focus on wrapper TSS techniques, where candidate subsets of training instances are evaluated using the SVM training accuracy. We consider five wrapper TSS strategies and show that those based on evolutionary approaches can significantly improve the accuracy of SVMs.     

Cycle-breaking acceleration for support vector regression

Support vector regression (SVR) is a powerful tool in modeling and prediction tasks with widespread application in many areas. The most representative algorithms to train SVR models are Shevade et al.'s Modification 2 and Lin's WSS1 and WSS2 methods in the LIBSVM library. Both are variants of standard SMO in which the updating pairs selected are those that most violate the Karush-Kuhn-Tucker optimality conditions, to which LIBSVM adds a heuristic to improve the decrease in the objective function. In this paper, and after presenting a simple derivation of the updating procedure based on a greedy maximization of the gain in the objective function, we show how cycle-breaking techniques that accelerate the convergence of support vector machines (SVM) in classification can also be applied under this framework, resulting in significantly improved training times for SVR.     

Possibilistic support vector machines

We propose new support vector machines (SVMs) that incorporate the geometric distribution of an input data set by associating each data point with a possibilistic membership, which measures the relative strength of the self class membership. By using a possibilistic distance measure based on the possibilistic membership, we reformulate conventional SVMs in three ways. The proposed methods are shown to have better classification performance than conventional SVMs in various tests.     

Prediction performance of support vector machines on input vector normalization methods

Support vector machines (SVM) based on the statistical learning theory is currently one of the most popular and efficient approaches for pattern recognition problem, because of their remarkable performance in terms of prediction accuracy. It is, however, required to choose a proper normalization method for input vectors in order to improve the system performance. Various normalization methods for SVMs have been studied in this research and the results showed that the normalization methods could affect the prediction performance. The results could be useful for determining a proper normalization method to achieve the best performance in SVMs.     

Color image watermark extraction based on support vector machines

This work proposes a novel watermarking technique called SVM-based Color Image Watermarking (SCIW), based on support vector machines (SVMs) for the authentication of color images. To protect the copyright of a color image, a signature (a watermark), which is represented by a sequence of binary data, is embedded in the color image. The watermark-extraction issue can be treated as a classification problem involving binary classes. The SCIW method constructs a set of training patterns with the use of binary labels by employing three image features, which are the differences between a local image statistic and the luminance value of the center pixel in a sliding window with three distinct shapes. This set of training patterns is gathered from a pair of images, an original image and its corresponding watermarked image in the spatial domain. A quasi-optimal hyperplane (a binary classifier) can be realized by an SVM. The SCIW method utilizes this set of training patterns to train the SVM and then applies the trained SVM to classify a set of testing patterns. Following the results produced by the classifier (the trained SVM), the SCIW method retrieves the hidden signature without the original image during watermark extraction. Experimental results have demonstrated that the SCIW method is sufficiently robust against several color-image manipulations, and that it outperforms other proposed methods considered in this work.     

Fuzzy rule-based support vector regression system

In this paper,we design a fuzzy rule-based support vector regression system.The proposed system utilizes the advantages of fuzzy model and support vector regression to extract support vectors to generate fuzzy if-then rules from the training data set.Based on the first-order linear Tagaki-Sugeno (TS) model,the structure of rules is identified by the support vector regression and then the consequent parameters of rules are tuned by the global least squares method.Our model is applied to the real world regression task.The simulation results gives promising performances in terms of a set of fuzzy rules,which can be easily interpreted by humans.     

Fuzzy nile-based support vector regression system

In this paper, we design a fuzzy rule-based support vector regression system. The proposed system utilizes the advantages of fuzzy model and support vector regression to extract support vectors to generate fuzzy if-then rules from the training data set. Based on the first-order hnear Tagaki-Sugeno （TS） model, the structure of rules is identified by the support vector regression and then the consequent parameters of rules are tuned by the global least squares method. Our model is applied to the real world regression task. The simulation results gives promising performances in terms of a set of fuzzy hales, which can be easily interpreted by humans.     

On-line independent support vector machines

Support vector machines (SVMs) are one of the most successful algorithms for classification. However, due to their space and time requirements, they are not suitable for on-line learning, that is, when presented with an endless stream of training observations.In this paper we propose a new on-line algorithm, called on-line independent support vector machines (OISVMs), which approximately converges to the standard SVM solution each time new observations are added; the approximation is controlled via a user-defined parameter. The method employs a set of linearly independent observations and tries to project every new observation onto the set obtained so far, dramatically reducing time and space requirements at the price of a negligible loss in accuracy. As opposed to similar algorithms, the size of the solution obtained by OISVMs is always bounded, implying a bounded testing time. These statements are supported by extensive experiments on standard benchmark databases as well as on two real-world applications, namely place recognition by a mobile robot in an indoor environment and human grasping posture classification.     

Fuzzy modelling via on-line support vector machines

This article introduces an approach to identify unknown nonlinear systems by fuzzy rules and support vector machines (SVMs). Structure identification is realised by an on-line SVM technique, the fuzzy rules are generated automatically. Time-varying learning rates are applied for updating the membership functions of the fuzzy rules. Finally, the upper bounds of the modelling errors are proven.     

Experimental study on parameter choices in norm-r support vector regression machines with noisy input

In [1], with the evidence framework, the almost inversely linear dependency between the optimal parameter r in norm-r support vector regression machine r-SVR and the Gaussian input noise is theoretically derived. When r takes a non-integer value, r-SVR cannot be easily realized using the classical QP optimization method. This correspondence attempts to achieve two goals: (1) The Newton-decent-method based implementation procedure of r-SVR is presented here; (2) With this procedure, the experimental studies on the dependency between the optimal parameter r in r-SVR and the Gaussian noisy input are given. Our experimental results here confirm the theoretical claim in [1].     

Texture classification using the support vector machines

In recent years, support vector machines (SVMs) have demonstrated excellent performance in a variety of pattern recognition problems. In this paper, we apply SVMs for texture classification, using translation-invariant features generated from the discrete wavelet frame transform. To alleviate the problem of selecting the right kernel parameter in the SVM, we use a fusion scheme based on multiple SVMs, each with a different setting of the kernel parameter. Compared to the traditional Bayes classifier and the learning vector quantization algorithm, SVMs, and, in particular, the fused output from multiple SVMs, produce more accurate classification results on the Brodatz texture album.     

Knowledge based Least Squares Twin support vector machines

We propose knowledge based versions of a relatively new family of SVM algorithms based on two non-parallel hyperplanes. Specifically, we consider prior knowledge in the form of multiple polyhedral sets and incorporate the same into the formulation of linear Twin SVM (TWSVM)/Least Squares Twin SVM (LSTWSVM) and term them as knowledge based TWSVM (KBTWSVM)/knowledge based LSTWSVM (KBLSTWSVM). Both of these formulations are capable of generating non-parallel hyperplanes based on real-world data and prior knowledge. We derive the solution of KBLSTWSVM and use it in our computational experiments for comparison against other linear knowledge based SVM formulations. Our experiments show that KBLSTWSVM is a versatile classifier whose solution is extremely simple when compared with other linear knowledge based SVM algorithms.     

A parsimony fuzzy rule-based classifier using axiomatic fuzzy set theory and support vector machines

This paper proposes a classification method that is based on easily interpretable fuzzy rules and fully capitalizes on the two key technologies, namely pruning the outliers in the training data by SVMs (support vector machines), i.e., eliminating the influence of outliers on the learning process; finding a fuzzy set with sound linguistic interpretation to describe each class based on AFS (axiomatic fuzzy set) theory. Compared with other fuzzy rule-based methods, the proposed models are usually more compact and easily understandable for the users since each class is described by much fewer rules. The proposed method also comes with two other advantages, namely, each rule obtained from the proposed algorithm is simply a conjunction of some linguistic terms, there are no parameters that are required to be tuned. The proposed classification method is compared with the previously published fuzzy rule-based classifiers by testing them on 16 UCI data sets. The results show that the fuzzy rule-based classifier presented in this paper, offers a compact, understandable and accurate classification scheme. A balance is achieved between the interpretability and the accuracy.     

Support vector machines with genetic fuzzy feature transformation for biomedical data classification

In this paper, we present a genetic fuzzy feature transformation method for support vector machines (SVMs) to do more accurate data classification. Given data are first transformed into a high feature space by a fuzzy system, and then SVMs are used to map data into a higher feature space and then construct the hyperplane to make a final decision. Genetic algorithms are used to optimize the fuzzy feature transformation so as to use the newly generated features to help SVMs do more accurate biomedical data classification under uncertainty. The experimental results show that the new genetic fuzzy SVMs have better generalization abilities than the traditional SVMs in terms of prediction accuracy.