基于最小二乘支持向量机的多属性决策

分析多属性决策问题, 提出了基于最小二乘支持向量机的多属性决策方法。该方法从决策问题本身出发,构造学习样本,再通过最小二乘支持向量机拟合出多属性效用函数,从而实现对方案的排序。该模型与支持向量机相比,参数少,算法简单、可靠。最后通过算例表明方法的可行性与有效性。     

基于无约束优化的最小二乘支撑向量机

为了解决最小二乘支撑向量机（LSSVM）优化问题需要耗费大量时间的问题,提出了利用牛顿优化法来解决LSSVM优化问题的方法（称为Newton-LSSVM）。首先把LSSVM优化问题转化为无约束化优化问题的形式,然后再采用牛顿优化法来迭代求解。实验结果表明,该方法在大幅度减少LSSVM算法的训练时间开销的同时,能够获得与采用传统优化方式求解LSSVM优化问题一样的泛化能力。     

用于多分类问题的最小二乘支持向量分类-回归机

基于支持向量机(SVM)的三分类方法是处理多分类问题的一类方法。提出了最小二乘支持向量分类〖CD*2〗回归机(LSSVCR)算法,通过最小二乘目标函数充分考虑所有样本点对分类的影响,使得训练集中即使有个别样本点被标错类别,对分类结果也不会产生太大的影响,从而提高分类的准确性。该方法能够提高分类的准确率和分类速度,同时算法对于不同类别间样本数目差异较大的情况也有很好的分类效果。数值实验结果表明所提算法是可行的,且与已有的三分类算法相比在分类准确性上平均提高了2.57%,在运算速度上也有了较大的提高。     

Text classification: A least square support vector machine approach

This paper presents a least square support vector machine (LS-SVM) that performs text classification of noisy document titles according to different predetermined categories. The system's potential is demonstrated with a corpus of 91,229 words from University of Denver's Penrose Library catalogue. The classification accuracy of the proposed LS-SVM based system is found to be over 99.9%. The final classifier is an LS-SVM array with Gaussian radial basis function (GRBF) kernel, which uses the coefficients generated by the latent semantic indexing algorithm for classification of the text titles. These coefficients are also used to generate the confidence factors for the inference engine that present the final decision of the entire classifier. The system is also compared with a K-nearest neighbor (KNN) and Naïve Bayes (NB) classifier and the comparison clearly claims that the proposed LS-SVM based architecture outperforms the KNN and NB based system. The comparison between the conventional linear SVM based classifiers and neural network based classifying agents shows that the LS-SVM with LSI based classifying agents improves text categorization performance significantly and holds a lot of potential for developing robust learning based agents for text classification.     

一种支持向量机的快速分类算法

鉴于传统支持向量机分类过程的计算量和支持向量的个数成正比,为了提高分类决策的速度,提出一种约简支持向量的快速分类算法,该算法对原始的支持向量进行特定比例的模糊均值聚类操作,按照分类误差最小的原则构建最小线性二乘回归模型,求解新的支持向量系数和决策函数的偏置.人造数据集和标准数据集上的实验表明,约简50%支持向量后,可以在保持分类精度在无统计意义的明显损失的前提下,使得分类速度提高50%.     

双隶属度模糊粗糙支持向量机

针对支持向量机方法处理不确定信息系统时存在的两个问题：一方面支持向量机训练对噪声样本敏感,另一方面支持向量机训练未考虑信息系统的不一致,利用模糊理论与粗糙集方法分别计算得到两种隶属度：模糊隶属度与粗糙隶属度,并将两种隶属度引入到标准支持向量机中得到一个新的支持向量机模型——双隶属度模糊粗糙支持向量机（DM-FRSVM）。分析该模型对于不确定问题的解决思路并进行对比研究,实验结果表明,在对于含有不确定信息的样本集进行分类时,DM-FRSVM表现出更好的推广性能。     

双隶属度模糊支持向量机算法

对现有的模糊支持向量机进行分析，提出一种改进的模糊支持向量机算法——双隶属度模糊支持向量机法（DM FSVM）。在传统的模糊支持向量机模型中，每一个训练样本的隶属函数中只有一个隶属度，而DM FSVM中每一个训练样本拥有两个隶属度。它既能保持传统模糊支持向量机的优点，又能充分利用有限样本，增加其分类推广能力。实验表明该算法较好地提高了分类精度。     

可保证分类性能的最小二乘支持向量机

当前支持向量机是分类研究与应用的一个热点。提出了一个新的最小二乘支持向量机算法,该算法向最小二乘支持向量机（LS-SVM）优化模型中融入了类内散度（VSLSVM）思想,即用优化准则Min w′Mw对原LS-SVM进行重组合,w为对应LS-SVM中的权向量,M是类内散度矩阵。提出的方法仅仅需要求解一个线性系统而不是凸规划问题,实验主要对SVM和Suykens等人的方法进行了比较,并验证了提出的算法的有效性。     

Sparse least square twin support vector machine with adaptive norm

By promoting the parallel hyperplanes to non-parallel ones in SVM, twin support vector machines (TWSVM) have attracted more attention. There are many modifications of them. However, most of the modifications minimize the loss function subject to the I ₂-norm or I ₁-norm penalty. These methods are non-adaptive since their penalty forms are fixed and pre-determined for any types of data. To overcome the above shortcoming, we propose l _p norm least square twin support vector machine (l _p LSTSVM). Our new model is an adaptive learning procedure with l _p -norm (0<p<1), where p is viewed as an adjustable parameter and can be automatically chosen by data. By adjusting the parameter p, l _p LSTSVM can not only select relevant features but also improve the classification accuracy. The solutions of the optimization problems in l _p LSTSVM are obtained by solving a series systems of linear equations (LEs) and the lower bounds of the solution is established which is extremely helpful for feature selection. Experiments carried out on several standard UCI data sets and synthetic data sets show the feasibility and effectiveness of the proposed method.     

ML-SLSTSVM: a new structural least square twin support vector machine for multi-label learning

Pattern Analysis and Applications - Multi-label learning (MLL) is a special supervised learning task, where any single instance possibly belongs to several classes simultaneously. Nowadays, MLL...     

Multivariate adaptive regression spline (MARS) and least squares support vector machine (LSSVM) for OCR prediction

This article investigates the feasibility of multivariate adaptive regression spline (MARS) and least squares support vector machine (LSSVM) for the prediction of over consolidation ratio (OCR) of clay deposits based on Piezocone Penetration Tests (PCPT) data. MARS uses piece-wise linear segments to describe the non-linear relationships between input and output variables. LSSVM is firmly based on the theory of statistical learning, and uses regression technique. The input parameters of the models are corrected cone resistance (q _t ), vertical total stress (σ_v), hydrostatic pore pressure (u ₀), pore pressure at the cone tip (u ₁), and the pore pressure just above the cone base (u ₂). The developed LSSVM model gives error bar of predicted OCR. Equations have also been developed for prediction of OCR. The performance of MARS and LSSVM models has been compared with the traditional methods for OCR prediction. As the results reveal, the proposed MARS and LSSVM models are robust models for determination of OCR.     

A decision based one-against-one method for multi-class support vector machine

The support vector machine (SVM) has a high generalisation ability to solve binary classification problems, but its extension to multi-class problems is still an ongoing research issue. Among the existing multi-class SVM methods, the one-against-one method is one of the most suitable methods for practical use. This paper presents a new multi-class SVM method that can reduce the number of hyperplanes of the one-against-one method and thus it returns fewer support vectors. The proposed algorithm works as follows. While producing the boundary of a class, no more hyperplanes are constructed if the discriminating hyperplanes of neighbouring classes happen to separate the rest of the classes. We present a large number of experiments that show that the training time of the proposed method is the least among the existing multi-class SVM methods. The experimental results also show that the testing time of the proposed method is less than that of the one-against-one method because of the reduction of hyperplanes and support vectors. The proposed method can resolve unclassifiable regions and alleviate the over-fitting problem in a much better way than the one-against-one method by reducing the number of hyperplanes. We also present a direct acyclic graph SVM (DAGSVM) based testing methodology that improves the testing time of the DAGSVM method.     

基于决策树对支持向量机的医学图像分类新方法

针对传统对支持向量机多类分类算法（Multi-TWSVM）中出现的模糊性问题,提出了一种基于遗传算法的决策树对支持向量机（GA-DTTSVM）多类分类算法。GA-DTTSVM用遗传算法对特征数据建立决策树,通过构建决策树可以分离样本的模糊区域,提高模糊区域样本的识别率。在决策树的每个节点上用对支持向量机（TWSVM）训练分类器,最后用训练的分类器进行分类和预测。实验结果表明,与决策树对支持向量机（DTTSVM）多类分类算法以及Multi-TWSVM相比,GA-DTTSVM多类分类算法具有较高的分类精度和较快的训练速度。     

Least square support vector machine and multivariate adaptive regression spline for modeling lateral load capacity of piles

This article adopts least square support vector machine (LSSVM) and multivariate adaptive regression spline (MARS) for prediction of lateral load capacity (Q) of pile foundation. LSSVM is firmly based on the theory of statistical learning, uses regression technique. MARS is a nonparametric regression technique that models complex relationships. Diameter of pile (D), depth of pile embedment (L), eccentricity of load (e), and undrained shear strength of soil (S _u) have been used as input parameters of LSSVM and MARS. Equations have been presented from the developed MARS and LSSVM. This study also presents a comparative study between the developed MARS and LSSVM.     

A decision support system for machine selection with load balancing

This paper presents a new approach towards the machine selection problem. This approach utilizes a decision support system (DSS) for advising the production manager which machine to select. The DSS is of special kind: its machine selection rules are modified to ensure equal load on the machines. The basic structure of the decision tree is defined by process experts and the rules threshold values are dynamically evaluated.

The implementation domain is the aluminum extrusion process, with the objective of aiding the manager in selecting the appropriate extrusion press machine from a set of similar machines. These machines are the most expensive equipment in the process, and are expected to be fully utilized.

The methodology presented in this paper was implemented in such a facility resulting in about 3% improvement on machine utilization.     

A new maximal-margin spherical-structured multi-class support vector machine

Support vector machines (SVMs), initially proposed for two-class classification problems, have been very successful in pattern recognition problems. For multi-class classification problems, the standard hyperplane-based SVMs are made by constructing and combining several maximal-margin hyperplanes, and each class of data is confined into a certain area constructed by those hyperplanes. Instead of using hyperplanes, hyperspheres that tightly enclosed the data of each class can be used. Since the class-specific hyperspheres are constructed for each class separately, the spherical-structured SVMs can be used to deal with the multi-class classification problem easily. In addition, the center and radius of the class-specific hypersphere characterize the distribution of examples from that class, and may be useful for dealing with imbalance problems. In this paper, we incorporate the concept of maximal margin into the spherical-structured SVMs. Besides, the proposed approach has the advantage of using a new parameter on controlling the number of support vectors. Experimental results show that the proposed method performs well on both artificial and benchmark datasets.     

Modeling medical decision making by support vector machines,explaining by rules of evolutionary algorithms with feature selection

Machine learning support for medical decision making is truly helpful only when it meets two conditions: high prediction accuracy and a good explanation of how the diagnosis was reached. Support vector machines (SVMs) successfully achieve the first target due to a kernel-based engine; evolutionary algorithms (EAs) can greatly accomplish the second owing to their adaptable nature. In this context, the current paper puts forward a two-step hybridized methodology, where learning is accurately performed by the SVMs and a comprehensible emulation of the resulting decision model is generated by EAs in the form of propositional rules, while referring only those indicators that highly influence the class separation. An individual highlighting of the medical attributes that trigger a specific diagnosis for a current patient record is additionally obtained; this feature thus increases the confidence of the physician in the resulting automated diagnosis. Without loss of generality, we aim to model three breast cancer instances, for reasons of both high incidence of the disease and the large application of state of the art artificial intelligence methods for this medical task. As such, the prediction of a benign/malignant condition as well as the recurrence/nonrecurrence of a cancer event are studied on the Wisconsin corresponding data sets from the UCI Machine Learning Repository. The proposed hybridization reached its goals. Rule prototypes evolve against a SVM consistent training data, while diversity among the different classes is implicitly preserved. Feature selection eventually leads to a resulting rule set where only the significant medical indicators together with the discriminating threshold values are referred, while individual relevance of attributes can be additionally obtained for each patient. The gain is thus dual: the EA benefits from a noise-free SVM preprocessed data and the resulting SVM model is able to output rules in a comprehensible, concise format for the physician.     

A bi-fuzzy progressive transductive support vector machine(BFPTSVM) algorithm

Transductive support vector machine (TSVM) is a well-known algorithm that realizes transductive learning in the field of support vector classification. This paper constructs a bi-fuzzy progressive transductive support vector machine (BFPTSVM) algorithm by combining the proposed notation of bi-fuzzy memberships for the temporary labeled sample appeared in progressive learning process and the sample-pruning strategy, which decreases the computation complexity and store memory of algorithm. Simulation experiments show that the BFPTSVM algorithm derives better classification performance and converges rapidly with better stability compared to the other learning algorithms.     

Parallel randomized sampling for support vector machine (SVM) and support vector regression (SVR)

A parallel randomized support vector machine (PRSVM) and a parallel randomized support vector regression (PRSVR) algorithm based on a randomized sampling technique are proposed in this paper. The proposed PRSVM and PRSVR have four major advantages over previous methods. (1) We prove that the proposed algorithms achieve an average convergence rate that is so far the fastest bounded convergence rate, among all SVM decomposition training algorithms to the best of our knowledge. The fast average convergence bound is achieved by a unique priority based sampling mechanism. (2) Unlike previous work (Provably fast training algorithm for support vector machines, 2001) the proposed algorithms work for general linear-nonseparable SVM and general non-linear SVR problems. This improvement is achieved by modeling new LP-type problems based on Karush–Kuhn–Tucker optimality conditions. (3) The proposed algorithms are the first parallel version of randomized sampling algorithms for SVM and SVR. Both the analytical convergence bound and the numerical results in a real application show that the proposed algorithm has good scalability. (4) We present demonstrations of the algorithms based on both synthetic data and data obtained from a real word application. Performance comparisons with SVM^light show that the proposed algorithms may be efficiently implemented.