A rough margin-based ν-twin support vector machine

Twin support vector machine (TSVM) is a new machine learning algorithm, which aims at finding two nonparallel planes for each class. In order to do so, one needs to resolve a pair of smaller-sized quadratic programming problems (QPPs) rather than a single large one. However, when constructing the classification plane for one class, a large number of samples of this class are considered in the objective function, but only fewer samples in the other class are considered, which easily results in over-fitting problem. In addition, the same penalties are given to each misclassified samples in the TSVM. In fact, the misclassified samples have different effects on the decision of the hyper-plane. In order to overcome these two disadvantages, by introducing the rough set theory into ν-TSVM, we propose a rough margin-based ν-TSVM in this paper. In the proposed algorithm, the different points in the different positions are proposed to have different effects on the separating hyper-plane. We firstly construct rough lower margin, rough upper margin, and rough boundary in the ν-TSVM and then give the different penalties to the different misclassified samples according to their positions. The new classifier can avoid the over-fitting problem to a certain extent. Numerical experiments on one artificial dataset and six benchmark datasets demonstrate the feasibility and validity of the proposed algorithm.     

An ε-twin support vector machine for regression

A special class of recurrent neural network termed Zhang neural network (ZNN) depicted in the implicit dynamics has recently been introduced for online solution of time-varying convex quadratic programming (QP) problems. Global exponential convergence of such a ZNN model is achieved theoretically in an error-free situation. This paper investigates the performance analysis of the perturbed ZNN model using a special type of activation functions (namely, power-sum activation functions) when solving the time-varying QP problems. Robustness analysis and simulation results demonstrate the superior characteristics of using power-sum activation functions in the context of large ZNN-implementation errors, compared with the case of using linear activation functions. Furthermore, the application to inverse kinematic control of a redundant robot arm also verifies the feasibility and effectiveness of the ZNN model for time-varying QP problems solving.     

ν-twin support vector machine with Universum data for classification

A novel ν-twin support vector machine with Universum data (\(\mathfrak {U}_{\nu }\)-TSVM) is proposed in this paper. \(\mathfrak {U}_{\nu }\)-TSVM allows to incorporate the prior knowledge embedded in the unlabeled samples into the supervised learning. It aims to utilize these prior knowledge to improve the generalization performance. Different from the conventional \(\mathfrak {U}\)-SVM, \(\mathfrak {U}_{\nu }\)-TSVM employs two Hinge loss functions to make the Universum data lie in a nonparallel insensitive loss tube, which makes it exploit these prior knowledge more flexibly. In addition, the newly introduced parameters ν₁, ν₂ in the \(\mathfrak {U}_{\nu }\)-TSVM have better theoretical interpretation than the penalty factor c in the \(\mathfrak {U}\)-TSVM. Numerical experiments on seventeen benchmark datasets, handwritten digit recognition, and gender classification indicate that the Universum indeed contributes to improving the prediction accuracy. Moreover, our \(\mathfrak {U}_{\nu }\)-TSVM is far superior to the other three algorithms (\(\mathfrak {U}\)-SVM, ν-TSVM and \(\mathfrak {U}\)-TSVM) from the prediction accuracy.     

An intelligent forecasting model based on robust wavelet ν-support vector machine

Aiming at the problem of small samples, season character, nonlinearity, randomicity and fuzziness in product demand series, the existing support vector kernel does not approach the random curve of the demands time series in the L²(Rⁿ) space (quadratic continuous integral space). The robust loss function is also proposed to solve the shortcoming of ε-insensitive loss function during handling hybrid noises. A novel robust wavelet support vector machine (RW ν-SVM) is proposed based on wavelet theory and the modified support vector machine. Particle swarm optimization algorithm is designed to select the optimal parameters of RW ν-SVM model in the scope of constraint permission. The results of application in car demand forecasts show that the forecasting approach based on the RW ν-SVM model is effective and feasible, the comparison between the method proposed in this paper and other ones is also given which proves this method is better than RW ν-SVM and other traditional methods.     

ν-Nonparallel support vector machine for pattern classification

In this paper, we propose a novel nonparallel hyperplane classifier, named ν-nonparallel support vector machine (ν-NPSVM), for binary classification. Based on our recently proposed method, i.e., nonparallel support vector machine (NPSVM), which has been proved superior to the twin support vector machines, ν-NPSVM is parameterized by the quantity ν to let ones effectively control the number of support vectors. By combining the ν-support vector classification and the ν-support vector regression together to construct the primal problems, ν-NPSVM inherits the advantages of ν-support vector machine so that enables us to eliminate one of the other free parameters of the NPSVM: the accuracy parameter ε and the regularization constant C. We describe the algorithm, give some theoretical results concerning the meaning and the choice of ν, and also report the experimental results on lots of data sets to show the effectiveness of our method.     

An improved robust and sparse twin support vector regression via linear programming

Twin support vector regression (TSVR) was proposed recently as a novel regressor that tries to find a pair of nonparallel planes, i.e. \(\epsilon \) -insensitive up- and down-bounds, by solving two related SVM-type problems. Though TSVR exhibits good performance compared with conventional methods like SVR, it suffers from the following issues: (1) it lacks model complexity control and thus may incur overfitting and suboptimal solution; (2) it needs to solve a pair of quadratic programming problems which are relatively complex to implement; (3) it is sensitive to outliers; and (4) its solution is not sparse. To address these problems, we propose in this paper a novel regression algorithm termed as robust and sparse twin support vector regression. The central idea is to reformulate TSVR as a convex problem by introducing regularization technique first and then derive a linear programming (LP) formulation which is not only simple but also allows robustness and sparseness. Instead of solving the resulting LP problem in the primal, we present a Newton algorithm with Armijo step-size to resolve the corresponding exact exterior penalty problem. The experimental results on several publicly available benchmark data sets show the feasibility and effectiveness of the proposed method.     

The forecasting model based on wavelet ν-support vector machine

Aiming at the series with small samples, seasonal character, nonlinearity, randomicity and fuzziness, the existing support vector kernel does not approach the random curve of the sales time series in the L²(Rⁿ) space (quadratic continuous integral space). A new wavelet support vector machine (WN ν-SVM) is proposed based on wavelet theory and modified support vector machine. A particle swarm optimization (PSO) algorithm is designed to select the best parameters of WN ν-SVM model in the scope of constraint permission. The results of application in car sale series forecasting show that the forecasting approach based on the PSOWN ν-SVM model is effective and feasible, the comparison between the method proposed in this paper and other ones is also given which proves this method is better than PSOW ν-SVM and other traditional methods.     

The forecasting model based on fuzzy novel ν-support vector machine

This paper presents a new version of fuzzy support vector machine to forecast multi-dimension fuzzy sample. By combining the triangular fuzzy theory with the modified ν-support vector machine, the fuzzy novel ν-support vector machine (FNν-SVM) is proposed, whose constraint conditions are less than those of the standard Fν-SVM by one, is proved to satisfy the structure risk minimum rule under the condition of probability. Moreover, there is no parameter b in the regression function of the FNν-SVM. To seek the optimal parameters of the FNν-SVM, particle swarm optimization is also proposed to optimize the unknown parameters of the FNν-SVM. The results of the application in sale forecasts confirm the feasibility and the validity of the FNν-SVM model. Compared with the traditional model, the FNν-SVM method requires fewer samples and has better forecasting precision.     

A rough <Emphasis Type="Italic">ν</Emphasis>-twin support vector regression machine

After combining the ν-Twin Support Vector Regression (ν-TWSVR) with the rough set theory, we propose an efficient Rough ν-Twin Support Vector Regression, called Rough ν-TWSVR for short. We construct a pair of optimization problems which are motivated by and mathematically derived from a related ν-TWSVR Rastogi et al. (Appl Intell 46(3):670–683 2017) and Rough ν-SVR Zhao et al. (Expert Syst Appl 36(6):9793–9798 2009). Rough ν-TWSVR not only utilizes more data information rather than the extreme data points in the ν-TWSVR, but also makes different points having different effects on the regressor depending on their positions. This method can implement the structural risk minimization and automatically control accuracies according to the structure of the data sets. In addition, the double ε s are utilized to construct the rough tube for upper(lower)-bound Rough ν-TWSVR instead of a single ε in the upper(lower)-bound ν-TWSVR. Moreover, This rough tube consisting of positive region, boundary region, and negative region yields the feasible set of the Rough ν-TWSVR larger than that of the ν-TWSVR, which makes the objective function of the Rough ν-TWSVR no more than that of ν-TWSVR. The Rough ν-TWSVR improves the generalization performance of the ν-TWSVR, especially for the data sets with outliers. Experimental results on toy examples and benchmark data sets confirm the validation and applicability of our proposed Rough ν-TWSVR.     

The incremental learning algorithm with support vector machine based on hyperplane-distance

This paper proves the problem of losing incremental samples’ information of the present SVM incremental learning algorithm from both theoretic and experimental aspects, and proposes a new incremental learning algorithm with support vector machine based on hyperplane-distance. According to the geometric character of support vector, the algorithm uses Hyperplane-Distance to extract the samples, selects samples which are most likely to become support vector to form the vector set of edge, and conducts the support vector machine training on the vector set. This method reduces the number of training samples and effectively improves training speed of incremental learning. The results of experiment performed on Chinese webpage classification show that this algorithm can reduce the number of training samples effectively and accumulate historical information. The HD-SVM algorithm has higher training speed and better precision of classification.     

Hybrid wavelet ν-support vector machine and chaotic particle swarm optimization for regression estimation

In view of the bad approximate results of the existing support vector (SV) kernel for series influenced by multi-factors in quadratic continuous integral space, combining wavelet theory with kernel technique, a wavelet kernel function is put forward in quadratic continuous integral space. And then, wavelet ν-support vector machine (W ν-SVM) with wavelet kernel is proposed. To seek the optimal parameters of W ν-SVM, embedded chaotic particle swarm optimization (ECPSO) is also proposed to optimize parameters of W ν-SVM. The results of application in car sale estimation show that the estimation approach based on the W ν-SVM and ECPSO is effective and feasible. Compared with the traditional model, W ν-SVM method requires fewer samples and has better estimating precision.     

Large data sets classification using convex–concave hull and support vector machine

Normal support vector machine (SVM) is not suitable for classification of large data sets because of high training complexity. Convex hull can simplify the SVM training. However, the classification accuracy becomes lower when there exist inseparable points. This paper introduces a novel method for SVM classification, called convex–concave hull SVM (CCH-SVM). After grid processing, the convex hull is used to find extreme points. Then, we use Jarvis march method to determine the concave (non-convex) hull for the inseparable points. Finally, the vertices of the convex–concave hull are applied for SVM training. The proposed CCH-SVM classifier has distinctive advantages on dealing with large data sets. We apply the proposed method on several benchmark problems. Experimental results demonstrate that our approach has good classification accuracy while the training is significantly faster than other SVM classifiers. Compared with the other convex hull SVM methods, the classification accuracy is higher.     

Fuzzy robust ν-support vector machine with penalizing hybrid noises on symmetric triangular fuzzy number space

In view of the shortage of ε-insensitive loss function for hybrid noises such as singularity points, biggish magnitude noises and Gaussian noises, this paper presents a new version of fuzzy support vector machine (SVM) which can penalize those hybrid noises to forecast fuzzy nonlinear system. Since there exist some problems of hybrid noises and uncertain data in many actual forecasting problem, the input variables are described as fuzzy numbers by fuzzy comprehensive evaluation. Then by the integration of the triangular fuzzy theory, ν-SVM and loss function theory, the fuzzy robust ν-SVM with robust loss function (FRν-SVM) which can penalize those hybrid noises is proposed. To seek the optimal parameters of FRν-SVM, particle swarm optimization is also proposed to optimize the unknown parameters of FRν-SVM. The results of the application in fuzzy sale system forecasts confirm the feasibility and the validity of the FRν-SVM model. Compared with the traditional model and other SVM methods, FRν-SVM method requires fewer samples and has better generalization capability for Gaussian noise.     

The complex fuzzy system forecasting model based on triangular fuzzy robust wavelet ν-support vector machine

This paper presents a new version of fuzzy wavelet support vector regression machine to forecast the nonlinear fuzzy system with multi-dimensional input variables. The input and output variables of the proposed model are described as triangular fuzzy numbers. Then by integrating the triangular fuzzy theory, wavelet analysis theory and ν-support vector regression machine, a polynomial slack variable is also designed, the triangular fuzzy robust wavelet ν-support vector regression machine (TFRWν-SVM) is proposed. To seek the optimal parameters of TFRWν-SVM, particle swarm optimization is also applied to optimize parameters of TFRWν-SVM. A forecasting method based on TFRWν-SVRM and PSO are put forward. The results of the application in sale system forecasts confirm the feasibility and the validity of the forecasting method. Compared with the traditional model, TFRWν-SVM method requires fewer samples and has better forecasting precision.     

A smart adaptive particle swarm optimization–support vector machine: android botnet detection application

The Journal of Supercomputing - Support vector machine (SVM) is a renowned machine learning technique, which has been successfully applied to solve many practical pattern classification problems....     

Joint feature re-extraction and classification using an iterative semi-supervised support vector machine algorithm

The focus of this paper is on joint feature re-extraction and classification in cases when the training data set is small. An iterative semi-supervised support vector machine (SVM) algorithm is proposed, where each iteration consists both feature re-extraction and classification, and the feature re-extraction is based on the classification results from the previous iteration. Feature extraction is first discussed in the framework of Rayleigh coefficient maximization. The effectiveness of common spatial pattern (CSP) feature, which is commonly used in Electroencephalogram (EEG) data analysis and EEG-based brain computer interfaces (BCIs), can be explained by Rayleigh coefficient maximization. Two other features are also defined using the Rayleigh coefficient. These features are effective for discriminating two classes with different means or different variances. If we extract features based on Rayleigh coefficient maximization, a large training data set with labels is required in general; otherwise, the extracted features are not reliable. Thus we present an iterative semi-supervised SVM algorithm embedded with feature re-extraction. This iterative algorithm can be used to extract these three features reliably and perform classification simultaneously in cases where the training data set is small. Each iteration is composed of two main steps: (i) the training data set is updated/augmented using unlabeled test data with their predicted labels; features are re-extracted based on the augmented training data set. (ii) The re-extracted features are classified by a standard SVM. Regarding parameter setting and model selection of our algorithm, we also propose a semi-supervised learning-based method using the Rayleigh coefficient, in which both training data and test data are used. This method is suitable when cross-validation model selection may not work for small training data set. Finally, the results of data analysis are presented to demonstrate the validity of our approach. Editor: Olivier Chapelle.     

Fuzzy nile-based support vector regression system

In this paper, we design a fuzzy rule-based support vector regression system. The proposed system utilizes the advantages of fuzzy model and support vector regression to extract support vectors to generate fuzzy if-then rules from the training data set. Based on the first-order hnear Tagaki-Sugeno （TS） model, the structure of rules is identified by the support vector regression and then the consequent parameters of rules are tuned by the global least squares method. Our model is applied to the real world regression task. The simulation results gives promising performances in terms of a set of fuzzy hales, which can be easily interpreted by humans.     

Spam detection on Twitter using a support vector machine and users’ features by identifying their interactions

Multimedia Tools and Applications - Spam tweets might cause numerous problems for users. An automatic method is introduced as a proposed method to detect spam tweets. This method is based on...     

A ν-twin support vector machine (ν-TSVM) classifier and its geometric algorithms

In this paper, a ν-twin support vector machine (ν-TSVM) is presented, improving upon the recently proposed twin support vector machine (TSVM). This ν-TSVM introduces a pair of parameters (ν) to control the bounds of the fractions of the support vectors and the error margins. The theoretical analysis shows that this ν-TSVM can be interpreted as a pair of minimum generalized Mahalanobis-norm problems on two reduced convex hulls (RCHs). Based on the well-known Gilbert’s algorithm, a geometric algorithm for TSVM (GA-TSVM) and its probabilistic speed-up version, named PGA-TSVM, are presented. Computational results on several synthetic as well as benchmark datasets demonstrate the significant advantages of the proposed algorithms in terms of both computation complexity and classification accuracy.     

Temperature prediction control based on least squares support vector machines

A prediction control algorithm is presented based on least squares support vector machines (LS-SVM) model for a class of complex systems with strong nonlinearity. The nonlinear off-line model of the controUed plant is built by LS-SVM with radial basis function (RBF) kernel. In the process of system running, the off-line model is linearized at each sampling instant, and the generalized prediction control (GPC) algorithm is employed to implement the prediction control for the controlled plant. The obtained algorithm is applied to a boiler temperature control system with complicated nonlinearity and large time delay. The results of the experiment verify the effectiveness and merit of the algorithm.