Least squares support vector regression for solving Volterra integral equations

In this paper, a numerical approach is proposed based on least squares support vector regression for solving Volterra integral equations of the first and second kind. The proposed method is based on using a hybrid of support vector regression with an orthogonal kernel and Galerkin and collocation spectral methods. An optimization problem is derived and transformed to solving a system of algebraic equations. The resulting system is discussed in terms of the structure of the involving matrices and the error propagation. Numerical results are presented to show the sparsity of resulting system as well as the efficiency of the method.

     

基于MapReduce的最小二乘支持向量机回归模型

针对最小二乘支持向量机处理大规模数据集耗时长且受内存限制的特点,将局部多模型方法与MapReduce编程模式相结合,提出一种并行最小二乘支持向量机回归模型.模型由两组MapReduce过程组成,首先按照输入样本集对样本数据进行聚类操作,再对聚类后得到的子类按输出样本集进行二次聚类操作,分别得到局部模型数目和各局部模型综合加权输出计算结果.实验结果表明,并行最小二乘支持向量机回归模型具有较好的加速比和可扩展性.     

Interval analysis using least squares support vector fuzzy regression

A least squares support vector fuzzy regression model(LS-SVFR) is proposed to estimate uncertain and imprecise data by applying the fuzzy set principle to weight vectors.This model only requires a set of linear equations to obtain the weight vector and the bias term,which is different from the solution of a complicated quadratic programming problem in existing support vector fuzzy regression models.Besides,the proposed LS-SVFR is a model-free method in which the underlying model function doesn’t need to be predefined.Numerical examples and fault detection application are applied to demonstrate the effectiveness and applicability of the proposed model.     

Intuitionistic fuzzy C-regression by using least squares support vector regression

This paper proposes a novel intuitionistic fuzzy c-least squares support vector regression (IFC-LSSVR) with a Sammon mapping clustering algorithm. Sammon mapping effectively reduces the complexity of raw data, while intuitionistic fuzzy sets (IFSs) can effectively tune the membership of data points, and LSSVR improves the conventional fuzzy c-regression model. The proposed clustering algorithm combines the advantages of IFSs, LSSVR and Sammon mapping for solving actual clustering problems. Moreover, IFC-LSSVR with Sammon mapping adopts particle swarm optimization to obtain optimal parameters. Experiments conducted on a web-based adaptive learning environment and a dataset of wheat varieties demonstrate that the proposed algorithm is more efficient than conventional algorithms, such as the k-means (KM) and fuzzy c-means (FCM) clustering algorithms, in standard measurement indexes. This study thus demonstrates that the proposed model is a credible fuzzy clustering algorithm. The novel method contributes not only to the theoretical aspects of fuzzy clustering, but is also widely applicable in data mining, image systems, rule-based expert systems and prediction problems.     

Interval type-2 fuzzy kernel based support vector machine algorithm for scene classification of humanoid robot

This paper proposed an Interval Type-2 Fuzzy Kernel based Support Vector Machine (IT2FK-SVM) for scene classification of humanoid robot. Type-2 fuzzy sets have been shown to be a more promising method to manifest the uncertainties. Kernel design is a key component for many kernel-based methods. By integrating the kernel design with type-2 fuzzy sets, a systematic design methodology of IT2FK-SVM classification for scene images is presented to improve robustness and selectivity in the humanoid robot vision, which involves feature extraction, dimensionality reduction and classifier learning. Firstly, scene images are represented as high dimensional vector extracted from intensity, edge and orientation feature maps by biological-vision feature extraction method. Furthermore, a novel three-domain Fuzzy Kernel-based Principal Component Analysis (3DFK-PCA) method is proposed to select the prominent variables from the high-dimensional scene image representation. Finally, an IT2FM SVM classifier is developed for the comprehensive learning of scene images in complex environment. Different noisy, different view angle, and variations in lighting condition can be taken as the uncertainties in scene images. Compare to the traditional SVM classifier with RBF kernel, MLP kernel, and the Weighted Kernel (WK), respectively, the proposed method performs much better than conventional WK method due to its integration of IT2FK, and WK method performs better than the single kernel methods (SVM classifier with RBF kernel or MLP kernel). IT2FK-SVM is able to deal with uncertainties when scene images are corrupted by various noises and captured by different view angles. The proposed IT2FK-SVM method yields over $92~\% $ classification rates for all cases. Moreover, it even achieves $98~\% $ classification rate on the newly built dataset with common light case.     

Regularized least squares fuzzy support vector regression for financial time series forecasting

In this paper, we propose a novel approach, termed as regularized least squares fuzzy support vector regression, to handle financial time series forecasting. Two key problems in financial time series forecasting are noise and non-stationarity. Here, we assign a higher membership value to data samples that contain more relevant information, where relevance is related to recency in time. The approach requires only a single matrix inversion. For the linear case, the matrix order depends only on the dimension in which the data samples lie, and is independent of the number of samples. The efficacy of the proposed algorithm is demonstrated on financial datasets available in the public domain.     

最小二乘双支持向量回归机

提出了一个最小二乘双支持向量回归机,它是在双支持向量回归机基础之上建立的,打破了标准支持向量回归机利用两条平行超平面构造ε带的思想。事实上,它是利用两条不一定平行的超平面构造ε带,每条超平面确定一个半ε-带,从而得到最终的回归函数,这使该回归函数更符合数据本身的分布情况,回归算法有更好的推广能力。另外,最小二乘双支持向量机只需求解两个较小规模的线性方程组就能得到最后的回归函数,其计算复杂度相对较低。数值实验也表明该回归算法在推广能力和计算效率上有一定的优势。     

Least squares twin parametric-margin support vector machine for classification

In this paper, we propose a novel least squares twin parametric-margin support vector machine (TPMSVM) for binary classification, called LSTPMSVM for short. LSTPMSVM attempts to solve two modified primal problems of TPMSVM, instead of two dual problems usually solved. The solution of the two modified primal problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems along with two systems of linear equations in TPMSVM, which leads to extremely simple and fast algorithm. Classification using nonlinear kernel with reduced technique also leads to systems of linear equations. Therefore our LSTPMSVM is able to solve large datasets accurately without any external optimizers. Further, a particle swarm optimization (PSO) algorithm is introduced to do the parameter selection. Our experiments on synthetic as well as on several benchmark data sets indicate that our LSTPMSVM has comparable classification accuracy to that of TPMSVM but with remarkably less computational time.     

Least squares twin support vector machines for pattern classification

In this paper we formulate a least squares version of the recently proposed twin support vector machine (TSVM) for binary classification. This formulation leads to extremely simple and fast algorithm for generating binary classifiers based on two non-parallel hyperplanes. Here we attempt to solve two modified primal problems of TSVM, instead of two dual problems usually solved. We show that the solution of the two modified primal problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems along with two systems of linear equations in TSVM. Classification using nonlinear kernel also leads to systems of linear equations. Our experiments on publicly available datasets indicate that the proposed least squares TSVM has comparable classification accuracy to that of TSVM but with considerably lesser computational time. Since linear least squares TSVM can easily handle large datasets, we further went on to investigate its efficiency for text categorization applications. Computational results demonstrate the effectiveness of the proposed method over linear proximal SVM on all the text corpuses considered.     

Primal least squares twin support vector regression

The training algorithm of classical twin support vector regression (TSVR) can be attributed to the solution of a pair of quadratic programming problems (QPPs) with inequality constraints in the dual space.However,this solution is affected by time and memory constraints when dealing with large datasets.In this paper,we present a least squares version for TSVR in the primal space,termed primal least squares TSVR (PLSTSVR).By introducing the least squares method,the inequality constraints of TSVR are transformed into equality constraints.Furthermore,we attempt to directly solve the two QPPs with equality constraints in the primal space instead of the dual space;thus,we need only to solve two systems of linear equations instead of two QPPs.Experimental results on artificial and benchmark datasets show that PLSTSVR has comparable accuracy to TSVR but with considerably less computational time.We further investigate its validity in predicting the opening price of stock.     

Recursive reduced least squares support vector regression

Combining reduced technique with iterative strategy, we propose a recursive reduced least squares support vector regression. The proposed algorithm chooses the data which make more contribution to target function as support vectors, and it considers all the constraints generated by the whole training set. Thus it acquires less support vectors, the number of which can be arbitrarily predefined, to construct the model with the similar generalization performance. In comparison with other methods, our algorithm also gains excellent parsimoniousness. Numerical experiments on benchmark data sets confirm the validity and feasibility of the presented algorithm. In addition, this algorithm can be extended to classification.     

Least squares support vector machines ensemble models for credit scoring

Due to recent financial crisis and regulatory concerns of Basel II, credit risk assessment is becoming one of the most important topics in the field of financial risk management. Quantitative credit scoring models are widely used tools for credit risk assessment in financial institutions. Although single support vector machines (SVM) have been demonstrated with good performance in classification, a single classifier with a fixed group of training samples and parameters setting may have some kind of inductive bias. One effective way to reduce the bias is ensemble model. In this study, several ensemble models based on least squares support vector machines (LSSVM) are brought forward for credit scoring. The models are tested on two real world datasets and the results show that ensemble strategies can help to improve the performance in some degree and are effective for building credit scoring models.     

Extended least squares support vector machines for ordinal regression

We extend LS-SVM to ordinal regression, which has wide applications in many domains such as social science and information retrieval where human-generated data play an important role. Most current methods based on SVM for ordinal regression suffer from the problem of ignoring the distribution information reflected by the samples clustered around the centers of each class. This problem would degrade the performance of SVM-based methods since the classifiers only depend on the scattered samples on the border which induce large margin. Our method takes the samples clustered around class centers into account and has a competitive computational complexity. Moreover, our method would easily produce the optimal cut-points according to the prior class probabilities and hence may obtain more reasonable results when the prior class probabilities are not the same. Experiments on simulated datasets and benchmark datasets, especially on the real ordinal datasets, demonstrate the effectiveness of our method.     

Least squares recursive projection twin support vector machine for classification

In this paper we formulate a least squares version of the recently proposed projection twin support vector machine (PTSVM) for binary classification. This formulation leads to extremely simple and fast algorithm, called least squares projection twin support vector machine (LSPTSVM) for generating binary classifiers. Different from PTSVM, we add a regularization term, ensuring the optimization problems in our LSPTSVM are positive definite and resulting better generalization ability. Instead of usually solving two dual problems, we solve two modified primal problems by solving two systems of linear equations whereas PTSVM need to solve two quadratic programming problems along with two systems of linear equations. Our experiments on publicly available datasets indicate that our LSPTSVM has comparable classification accuracy to that of PTSVM but with remarkably less computational time.     

A new support vector machine based on type-2 fuzzy samples

Classical support vector machine is based on the real valued random samples and established on the probability space. It is hard to deal with classification problems based on type-2 fuzzy samples established on non-probability space. The existing algorithm, type-2 fuzzy support vector machine established on generalized credibility space, transforms the classification problems based on type-2 fuzzy samples to general fuzzy optimization problems and expands the application range of traditional support vector machine. However, nonnegativeness of the decision variables of general fuzzy optimization problems is too strict to be satisfied in some practical applications. Motivated by this, the concept of expected fuzzy possibility measure is proposed. Then type-2 fuzzy support vector machine on expected fuzzy possibility space is established, and the second-order cone programming of type-2 fuzzy support vector machine on expected fuzzy possibility space is given. The results of numerical experiments show the effectiveness of the type-2 fuzzy support vector machine established on expected fuzzy possibility space.     

Uncertainty measures for interval type-2 fuzzy sets

Fuzziness (entropy) is a commonly used measure of uncertainty for type-1 fuzzy sets. For interval type-2 fuzzy sets (IT2 FSs), centroid, cardinality, fuzziness, variance and skewness are all measures of uncertainties. The centroid of an IT2 FS has been defined by Karnik and Mendel. In this paper, the other four concepts are defined. All definitions use a Representation Theorem for IT2 FSs. Formulas for computing the cardinality, fuzziness, variance and skewness of an IT2 FS are derived. These definitions should be useful in IT2 fuzzy logic systems design using the principles of uncertainty, and in measuring the similarity between two IT2 FSs.     

A wavelet-based image denoising using least squares support vector machine

The least squares support vector machine (LS-SVM) is a modified version of SVM, which uses the equality constraints to replace the original convex quadratic programming problem. Consequently, the global minimizer is much easier to obtain in LS-SVM by solving the set of linear equation. LS-SVM has shown to exhibit excellent classification performance in many applications. In this paper, a wavelet-based image denoising using LS-SVM is proposed. Firstly, the noisy image is decomposed into different subbands of frequency and orientation responses using the wavelet transform. Secondly, the feature vector for a pixel in a noisy image is formed by the spatial regularity in wavelet domain, and the LS-SVM model is obtained by training. Then the wavelet coefficients are divided into two classes (noisy coefficients and noise-free ones) by LS-SVM training model. Finally, all noisy wavelet coefficients are relatively well denoised by soft-thresholding method. Extensive experimental results demonstrate that our method can obtain better performances in terms of both subjective and objective evaluations than those state-of-the-art denoising techniques. Especially, the proposed method can preserve edges very well while removing noise.     

一种快速稀疏最小二乘支持向量回归机

将Jiao法直接应用于最小二乘支持向量回归机上的效果并不理想,为此采用不完全抛弃的策略,提出了改进的Jiao法,并将其应用于最小二乘支持向量回归机.数据集测试的结果表明,基于改进Jiao法的稀疏最小二乘支持向量回归机,无论在支持向量个数和训练时间上都取得了一定的优势.与其他剪枝算法相比,在不丧失回归精度的情况下,改进的Jiao法可大大缩短训练时间.另外,改进的Jiao法同样适用于分类问题.