Smooth relevance vector machine: a smoothness prior extension of the RVM

Enforcing sparsity constraints has been shown to be an effective and efficient way to obtain state-of-the-art results in regression and classification tasks. Unlike the support vector machine (SVM) the relevance vector machine (RVM) explicitly encodes the criterion of model sparsity as a prior over the model weights. However the lack of an explicit prior structure over the weight variances means that the degree of sparsity is to a large extent controlled by the choice of kernel (and kernel parameters). This can lead to severe overfitting or oversmoothing—possibly even both at the same time (e.g. for the multiscale Doppler data). We detail an efficient scheme to control sparsity in Bayesian regression by incorporating a flexible noise-dependent smoothness prior into the RVM. We present an empirical evaluation of the effects of choice of prior structure on a selection of popular data sets and elucidate the link between Bayesian wavelet shrinkage and RVM regression. Our model encompasses the original RVM as a special case, but our empirical results show that we can surpass RVM performance in terms of goodness of fit and achieved sparsity as well as computational performance in many cases. The code is freely available. Action Editor: Dale Schuurmans.     

Classification of silent speech using support vector machine and relevance vector machine

To provide speech prostheses for individuals with severe communication impairments, we investigated a classification method for brain computer interfaces (BCIs) using silent speech. Event-related potentials (ERPs) were recorded using scalp electrodes when five subjects imagined the vocalization of Japanese vowels, /a/, /i/, /u/, /e/, and /o/ in order and in random order, while the subjects remained silent and immobilized.For actualization, we tried to apply relevance vector machine (RVM) and RVM with Gaussian kernel (RVM-G) instead of support vector machine with Gaussian kernel (SVM-G) to reduce the calculation cost in the use of 19 channels, common special patterns (CSPs) filtering, and adaptive collection (AC). Results show that using RVM-G instead of SVM-G reduced the ratio of the number of efficient vectors to the number of training data from 97% to 55%. At this time, the averaged classification accuracies (CAs) using SVM-G and RVM-G were, respectively, 77% and 79%, showing no degradation. However, the calculation cost was more than that using SVM-G because RVM-G necessitates high calculation costs for optimization. Furthermore, results show that CAs using RVM-G were weaker than SVM-G when the training data were few. Additionally, results showed that nonlinear classification was necessary for silent speech classification.This paper serves as a beginning of feasibility study for speech prostheses using an imagined voice. Although classification for silent speech presents great potential, many feasibility problems remain.     

支持向量机和最小二乘支持向量机的比较及应用研究

介绍和比较了支持向量机分类器和量小二乘支持向量机分类器的算法。并将支持向量机分类器和量小二乘支持向量机分类器应用于心脏病诊断，取得了较高的准确率。所用数据来自UCI bench—mark数据集。实验结果表明，支持向量机和量小二乘支持向量机在医疗诊断中有很大的应用潜力。     

Combining wavelet-based feature extractions with relevance vector machines for stock index forecasting

Abstract: The relevance vector machine (RVM) is a Bayesian version of the support vector machine, which with a sparse model representation has appeared to be a powerful tool for time-series forecasting. The RVM has demonstrated better performance over other methods such as neural networks or autoregressive integrated moving average based models. This study proposes a hybrid model that combines wavelet-based feature extractions with RVM models to forecast stock indices. The time series of explanatory variables are decomposed using some wavelet bases and the extracted time-scale features serve as inputs of an RVM to perform the non-parametric regression and forecasting. Compared with traditional forecasting models, our proposed method performs best. The root-mean-squared forecasting errors are significantly reduced.     

Optimal feature selection for support vector machines

Selecting relevant features for support vector machine (SVM) classifiers is important for a variety of reasons such as generalization performance, computational efficiency, and feature interpretability. Traditional SVM approaches to feature selection typically extract features and learn SVM parameters independently. Independently performing these two steps might result in a loss of information related to the classification process. This paper proposes a convex energy-based framework to jointly perform feature selection and SVM parameter learning for linear and non-linear kernels. Experiments on various databases show significant reduction of features used while maintaining classification performance.     

基于支持向量机的一类水域叶绿素a浓度反演研究

指出了用叶绿素a的浓度估计海洋初级生产力的重要作用;分析了目前采用的浓度反演方法的不足;尝试将基于统计学习理论的最小二乘支持向量机用于浓度反演, SeaBAM的数据实验结果表明该方法可以获得更高的反演精度;可以有效避免过学习的情况出现;不像神经网络那样需要确定网络结构。     

Human motion recognition using support vector machines

We propose a motion recognition strategy that represents each videoclip by a set of filtered images, each of which corresponds to a frame. Using a filtered-image classifier based on support vector machines, we classify a videoclip by applying majority voting over the predicted labels of its filtered images and, for online classification, we identify the most likely type of action at any moment by applying majority voting over the predicted labels of the filtered images within a sliding window. We also define a classification confidence and the associated threshold in both cases, which enable us to identify the existence of an unknown type of motion and, together with the proposed recognition strategy, make it possible to build a real-time motion recognition system that cannot only make classifications in real-time, but also learn new types of motions and recognize them in the future. The proposed strategy is demonstrated on real datasets.     

A note on Platt’s probabilistic outputs for support vector machines

Platt’s probabilistic outputs for Support Vector Machines (Platt, J. in Smola, A., et al. (eds.) Advances in large margin classifiers. Cambridge, 2000) has been popular for applications that require posterior class probabilities. In this note, we propose an improved algorithm that theoretically converges and avoids numerical difficulties. A simple and ready-to-use pseudo code is included. Editor: Dale Schuurmans.     

Fuzzy functions with support vector machines

A new fuzzy system modeling (FSM) approach that identifies the fuzzy functions using support vector machines (SVM) is proposed. This new approach is structurally different from the fuzzy rule base approaches and fuzzy regression methods. It is a new alternate version of the earlier FSM with fuzzy functions approaches. SVM is applied to determine the support vectors for each fuzzy cluster obtained by fuzzy c-means (FCM) clustering algorithm. Original input variables, the membership values obtained from the FCM together with their transformations form a new augmented set of input variables. The performance of the proposed system modeling approach is compared to previous fuzzy functions approaches, standard SVM, LSE methods using an artificial sparse dataset and a real-life non-sparse dataset. The results indicate that the proposed fuzzy functions with support vector machines approach is a feasible and stable method for regression problems and results in higher performances than the classical statistical methods.     

Prediction model building with clustering-launched classification and support vector machines in credit scoring

Recently, credit scoring has become a very important task as credit cards are now widely used by customers. A method that can accurately predict credit scoring is greatly needed and good prediction techniques can help to predict credit more accurately. One powerful classifier, the support vector machine (SVM), was successfully applied to a wide range of domains. In recent years, researchers have applied the SVM-based in the prediction of credit scoring, and the results have been shown it to be effective. In this study, two real world credit datasets in the University of California Irvine Machine Learning Repository were selected. SVM and a new classifier, clustering-launched classification (CLC), were employed to predict the accuracy of credit scoring. The advantages of using CLC are that it can classify data efficiently and only need one parameter needs to be decided. In substance, the results show that CLC is better than SVM. Therefore, CLC is an effective tool to predict credit scoring.     

一种基于SVM的相关反馈图像检索算法

相关反馈技术是近年来图像检索中的研究热点,本文以MPEG-7的边缘直方图作为图像特征,以支持向量机（SVM）为分类器,提出一种新的相关反馈算法。在每次反馈中对用户标记的相关样本进行学习,用历次返回的结果更新训练样本集,建立SVM分类器模型,并根据模型进行检索。本文还对不同核函数的SVM进行了对比,得出RBF核函数的SVM有较高的检索精度。使用由10000幅图像组成的图像库进行实验,结果表明,该算法可有效地检索出更多的相关图像,并且在有限训练样本情况下具有良好的泛化能力。     

Density-induced margin support vector machines

This paper proposes a new classifier called density-induced margin support vector machines (DMSVMs). DMSVMs belong to a family of SVM-like classifiers. Thus, DMSVMs inherit good properties from support vector machines (SVMs), e.g., unique and global solution, and sparse representation for the decision function. For a given data set, DMSVMs require to extract relative density degrees for all training data points. These density degrees can be taken as relative margins of corresponding training data points. Moreover, we propose a method for estimating relative density degrees by using the K nearest neighbor method. We also show the upper bound on the leave-out-one error of DMSVMs for a binary classification problem and prove it. Promising results are obtained on toy as well as real-world data sets.     

使用超椭球参数化坐标的支持向量机

基于n维超椭球面坐标变换公式,构造一类核函数--n维超椭球坐标变换核.由于是同维映射,且增大了类间距离,这类核函数在一定程度上改善了支持向量机的性能.与其他核函数(如高斯核)相比,将所构造的核函数用于支持向量机,仅产生了很少的支持向量,因而大大加快了学习速度,改善了泛化性能.数值实验结果表明了所构造的核函数的有效性和正确性.     

Short-term fault prediction based on support vector machines with parameter optimization by evolution strategy

Support vector machines (SVMs) are the effective machine-learning methods based on the structural risk minimization (SRM) principle, which is an approach to minimize the upper bound risk functional related to the generalization performance. The parameter selection is an important factor that impacts the performance of SVMs. Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) is an evolutionary optimization strategy, which is used to optimize the parameters of SVMs in this paper. Compared with the traditional SVMs, the optimal SVMs using CMA-ES have more accuracy in predicting the Lorenz signal. The industry case illustrates that the proposed method is very successfully in forecasting the short-term fault of large machinery.     

Three-parameter sequential minimal optimization for support vector machines

The well-known sequential minimal optimization (SMO) algorithm is the most commonly used algorithm for numerical solutions of the support vector learning problems. At each iteration in the traditional SMO algorithm, also called 2PSMO algorithm in this paper, it jointly optimizes only two chosen parameters. The two parameters are selected either heuristically or randomly, whilst the optimization with respect to the two chosen parameters is performed analytically. The 2PSMO algorithm is naturally generalized to the three-parameter sequential minimal optimization (3PSMO) algorithm in this paper. At each iteration of this new algorithm, it jointly optimizes three chosen parameters. As in 2PSMO algorithm, the three parameters are selected either heuristically or randomly, whilst the optimization with respect to the three chosen parameters is performed analytically. Consequently, the main difference between these two algorithms is that the optimization is performed at each iteration of the 2PSMO algorithm on a line segment, whilst that of the 3PSMO algorithm on a two-dimensional region consisting of infinitely many line segments. This implies that the maximum can be attained more efficiently by 3PSMO algorithm. Main updating formulae of both algorithms for each support vector learning problem are presented. To assess the efficiency of the 3PSMO algorithm compared with the 2PSMO algorithm, 14 benchmark datasets, 7 for classification and 7 for regression, will be tested and numerical performances are compared. Simulation results demonstrate that the 3PSMO outperforms the 2PSMO algorithm significantly in both executing time and computation complexity.     

Automatic detection of breast cancers in mammograms using structured support vector machines

Breast cancer is one of the most common cancers diagnosed in women. Large margin classifiers like the support vector machine (SVM) have been reported effective in computer-assisted diagnosis systems for breast cancers. However, since the separating hyperplane determination exclusively relies on support vectors, the SVM is essentially a local classifier and its performance can be further improved. In this work, we introduce a structured SVM model to determine if each mammographic region is normal or cancerous by considering the cluster structures in the training set. The optimization problem in this new model can be solved efficiently by being formulated as one second order cone programming problem. Experimental evaluation is performed on the Digital Database for Screening Mammography (DDSM) dataset. Various types of features, including curvilinear features, texture features, Gabor features, and multi-resolution features, are extracted from the sample images. We then select the salient features using the recursive feature elimination algorithm. The structured SVM achieves better detection performance compared with a well-tested SVM classifier in terms of the area under the ROC curve.     

一种新的基于SVM的相关反馈图像检索算法

提出了一种新的基于支持向量机(SVM)的相关反馈图像检索算法。实验结果表明,该算法在一定程度上解决了基于SVM的相关反馈图像检索中存在的样本不足的困难,提高了系统的检索性能。     

Relevance vector machine based infinite decision agent ensemble learning for credit risk analysis

In this paper, a relevance vector machine based infinite decision agent ensemble learning (RVM_Ideal) system is proposed for the robust credit risk analysis. In the first level of our model, we adopt soft margin boosting to overcome overfitting. In the second level, the RVM algorithm is revised for boosting so that different RVM agents can be generated from the updated instance space of the data. In the third level, the perceptron Kernel is employed in RVM to simulate infinite subagents. Our system RVM_Ideal also shares some good properties, such as good generalization performance, immunity to overfitting and predicting the distance to default. According to the experimental results, our proposed system can achieve better performance in term of sensitivity, specificity and overall accuracy.