Analysis of electrocardiographic changes in partial epileptic patients by combining eigenvector methods and support vector machines

Abstract: In the present study, the diagnostic accuracy of support vector machines (SVMs) on electrocardiogram (ECG) signals is evaluated. Two types of ECG beats (normal and partial epilepsy) were obtained from the Physiobank database. Decision making was performed in two stages: feature extraction by eigenvector methods and classification using the SVM trained on the extracted features. The present research demonstrates that the power levels of the power spectral densities obtained by eigenvector methods are features which represent the ECG signals well and SVMs trained on these features achieve high classification accuracies.     

The analysis of decomposition methods for support vector machines

The support vector machine (SVM) is a promising technique for pattern recognition. It requires the solution of a large dense quadratic programming problem. Traditional optimization methods cannot be directly applied due to memory restrictions. Up to now, very few methods can handle the memory problem and an important one is the "decomposition method." However, there is no convergence proof so far. We connect this method to projected gradient methods and provide theoretical proofs for a version of decomposition methods. An extension to bound-constrained formulation of SVM is also provided. We then show that this convergence proof is valid for general decomposition methods if their working set selection meets a simple requirement.     

A comparison of methods for multiclass support vector machines

Support vector machines (SVMs) were originally designed for binary classification. How to effectively extend it for multiclass classification is still an ongoing research issue. Several methods have been proposed where typically we construct a multiclass classifier by combining several binary classifiers. Some authors also proposed methods that consider all classes at once. As it is computationally more expensive to solve multiclass problems, comparisons of these methods using large-scale problems have not been seriously conducted. Especially for methods solving multiclass SVM in one step, a much larger optimization problem is required so up to now experiments are limited to small data sets. In this paper we give decomposition implementations for two such "all-together" methods. We then compare their performance with three methods based on binary classifications: "one-against-all," "one-against-one," and directed acyclic graph SVM (DAGSVM). Our experiments indicate that the "one-against-one" and DAG methods are more suitable for practical use than the other methods. Results also show that for large problems methods by considering all data at once in general need fewer support vectors.     

多类支持向量机方法的研究现状与分析

支持向量机（SVM）是建立在统计学理论基础上的一种小样本机器学习方法，最初应用于解决两类分类问题．然而在解决实际问题中遇到的多为多分类问题，如何有效的将其推广到多类分类问题是一个正在研究的问题．该文对现有的多类支持向量机方法从组合多个两类分类器、层次结构、一次性优化问题和纠错编码等4个角度进行了综合归纳和分析，详细介绍了每种方法的代表性算法，并比较其优劣．     

Combining meta-learning and search techniques to select parameters for support vector machines

Support Vector Machines (SVMs) have achieved very good performance on different learning problems. However, the success of SVMs depends on the adequate choice of the values of a number of parameters (e.g., the kernel and regularization parameters). In the current work, we propose the combination of meta-learning and search algorithms to deal with the problem of SVM parameter selection. In this combination, given a new problem to be solved, meta-learning is employed to recommend SVM parameter values based on parameter configurations that have been successfully adopted in previous similar problems. The parameter values returned by meta-learning are then used as initial search points by a search technique, which will further explore the parameter space. In this proposal, we envisioned that the initial solutions provided by meta-learning are located in good regions of the search space (i.e. they are closer to optimum solutions). Hence, the search algorithm would need to evaluate a lower number of candidate solutions when looking for an adequate solution. In this work, we investigate the combination of meta-learning with two search algorithms: Particle Swarm Optimization and Tabu Search. The implemented hybrid algorithms were used to select the values of two SVM parameters in the regression domain. These combinations were compared with the use of the search algorithms without meta-learning. The experimental results on a set of 40 regression problems showed that, on average, the proposed hybrid methods obtained lower error rates when compared to their components applied in isolation.     

SMO-based pruning methods for sparse least squares support vector machines

Solutions of least squares support vector machines (LS-SVMs) are typically nonsparse. The sparseness is imposed by subsequently omitting data that introduce the smallest training errors and retraining the remaining data. Iterative retraining requires more intensive computations than training a single nonsparse LS-SVM. In this paper, we propose a new pruning algorithm for sparse LS-SVMs: the sequential minimal optimization (SMO) method is introduced into pruning process; in addition, instead of determining the pruning points by errors, we omit the data points that will introduce minimum changes to a dual objective function. This new criterion is computationally efficient. The effectiveness of the proposed method in terms of computational cost and classification accuracy is demonstrated by numerical experiments.     

A study on SMO-type decomposition methods for support vector machines

Decomposition methods are currently one of the major methods for training support vector machines. They vary mainly according to different working set selections. Existing implementations and analysis usually consider some specific selection rules. This paper studies sequential minimal optimization type decomposition methods under a general and flexible way of choosing the two-element working set. The main results include: 1) a simple asymptotic convergence proof, 2) a general explanation of the shrinking and caching techniques, and 3) the linear convergence of the methods. Extensions to some support vector machine variants are also discussed.     

Prediction performance of support vector machines on input vector normalization methods

Support vector machines (SVM) based on the statistical learning theory is currently one of the most popular and efficient approaches for pattern recognition problem, because of their remarkable performance in terms of prediction accuracy. It is, however, required to choose a proper normalization method for input vectors in order to improve the system performance. Various normalization methods for SVMs have been studied in this research and the results showed that the normalization methods could affect the prediction performance. The results could be useful for determining a proper normalization method to achieve the best performance in SVMs.     

Classification of electrocardiogram signals with support vector machines and extreme learning machine

An Electrocardiogram or ECG is an electrical recording of the heart and is used in the investigation of heart disease. This ECG can be classified as normal and abnormal signals. The classification of the ECG signals is presently performed with the support vector machine. The generalization performance of the SVM classifier is not sufficient for the correct classification of ECG signals. To overcome this problem, the ELM classifier is used which works by searching for the best value of the parameters that tune its discriminant function and upstream by looking for the best subset of features that feed the classifier. The experiments were conducted on the ECG data from the Physionet arrhythmia database to classify five kinds of abnormal waveforms and normal beats. In this paper, a thorough experimental study was done to show the superiority of the generalization capability of the Extreme Learning Machine (ELM) that is presented and compared with support vector machine (SVM) approach in the automatic classification of ECG beats. In particular, the sensitivity of the ELM classifier is tested and that is compared with SVM combined with two classifiers, and they are the k-nearest Neighbor Classifier and the radial basis function neural network classifier, with respect to the curse of dimensionality and the number of available training beats. The obtained results clearly confirm the superiority of the ELM approach as compared with traditional classifiers.     

Twin support vector machines and subspace learning methods for microcalcification clusters detection

This paper presents a novel framework for microcalcification clusters (MCs) detection in mammograms. The proposed framework has three main parts: (1) first, MCs are enhanced by using a simple-but-effective artifact removal filter and a well-designed high-pass filter; (2) thereafter, subspace learning algorithms can be embedded into this framework for subspace (feature) selection of each image block to be handled; and (3) finally, in the resulted subspaces, the MCs detection procedure is formulated as a supervised learning and classification problem, and in this work, the twin support vector machine (TWSVM) is developed in decision-making of MCs detection. A large number of experiments are carried out to evaluate and compare the MCs detection approaches, and the effectiveness of the proposed framework is well demonstrated.     

Expert systems for time-varying biomedical signals using eigenvector methods

In this paper, we present the expert systems for time-varying biomedical signals classification and determine their accuracies. The combined neural network (CNN), mixture of experts (ME), and modified mixture of experts (MME) were tested and benchmarked for their performance on the classification of the studied time-varying biomedical signals (ophthalmic arterial Doppler signals, internal carotid arterial Doppler signals and electroencephalogram signals). Decision making was performed in two stages: feature extraction by eigenvector methods and classification using the classifiers trained on the extracted features. The inputs of these expert systems composed of diverse or composite features were chosen according to the network structures. The present study was conducted with the purpose of answering the question of whether the expert system with diverse features (MME) or composite feature (CNN, ME) improve the capability of classification of the time-varying biomedical signals. The purpose was to determine an optimum classification scheme for the problem and also to infer clues about the extracted features. Our research demonstrated that the power levels of power spectral density (PSD) estimations obtained by the eigenvector methods are the valuable features which are representing the time-varying biomedical signals and the CNN, ME, and MME trained on these features achieved high classification accuracies.     

Progressive refinement for support vector machines

Support vector machines (SVMs) have good accuracy and generalization properties, but they tend to be slow to classify new examples. In contrast to previous work that aims to reduce the time required to fully classify all examples, we present a method that provides the best-possible classification given a specific amount of computational time. We construct two SVMs: a “full” SVM that is optimized for high accuracy, and an approximation SVM (via reduced-set or subset methods) that provides extremely fast, but less accurate, classifications. We apply the approximate SVM to the full data set, estimate the posterior probability that each classification is correct, and then use the full SVM to reclassify items in order of their likelihood of misclassification. Our experimental results show that this method rapidly achieves high accuracy, by selectively devoting resources (reclassification) only where needed. It also provides the first such progressive SVM solution that can be applied to multiclass problems.     

Support vector machines for olfactory signals recognition

Pattern recognition techniques have widely been used in the context of odor recognition. The recognition of mixtures and simple odors as separate clusters is an untractable problem with some of the classical supervised methods. Recently, a new paradigm has been introduced in which the detection problem can be seen as a learning from examples problem. In this paper, we investigate odor recognition in this new perspective and in particular by using a novel learning scheme known as support vector machines (SVM) which guarantees high generalization ability on the test set. We illustrate the basics of the theory of SVM and show its performance in comparison with radial basis network and the error backpropagation training method. The leave-one-out procedure has been used for all classifiers, in order to finding the near-optimal SVM parameter and both to reduce the generalization error and to avoid outliers.     

改进的支持向量机分类算法

在研究了标准SVM分类算法后,本文提出了一种快速的支持向量机分类方法.该方法通过解决两类相关的SVM问题,找到两个非平行的平面,其中每个平面靠近其相应的类样本点,远离另一类样本点,最后通过这两个平面找到一个将两类样本分开的最优平面.在处理非线性情况下,引入一种快速核函数分类方法.使用该算法可以使分类的速度得到很大提高,针对实际数据集的实验表明了该算法的有效性.     

Help-Training for semi-supervised support vector machines

In this paper, we propose to reinforce the Self-Training strategy in semi-supervised mode by using a generative classifier that may help to train the main discriminative classifier to label the unlabeled data. We call this semi-supervised strategy Help-Training and apply it to training kernel machine classifiers as support vector machines (SVMs) and as least squares support vector machines. In addition, we propose a model selection strategy for semi-supervised training. Experimental results on both artificial and real problems demonstrate that Help-Training outperforms significantly the standard Self-Training. Moreover, compared to other semi-supervised methods developed for SVMs, our Help-Training strategy often gives the lowest error rate.     

Successive overrelaxation for support vector machines

Successive overrelaxation (SOR) for symmetric linear complementarity problems and quadratic programs is used to train a support vector machine (SVM) for discriminating between the elements of two massive datasets, each with millions of points. Because SOR handles one point at a time, similar to Platt's sequential minimal optimization (SMO) algorithm (1999) which handles two constraints at a time and Joachims' SVM(light) (1998) which handles a small number of points at a time, SOR can process very large datasets that need not reside in memory. The algorithm converges linearly to a solution. Encouraging numerical results are presented on datasets with up to 10 000 000 points. Such massive discrimination problems cannot be processed by conventional linear or quadratic programming methods, and to our knowledge have not been solved by other methods. On smaller problems, SOR was faster than SVM(light) and comparable or faster than SMO.     

Information criteria for support vector machines

This paper presents kernel regularization information criterion (KRIC), which is a new criterion for tuning regularization parameters in kernel logistic regression (KLR) and support vector machines (SVMs). The main idea of the KRIC is based on the regularization information criterion (RIC). We derive an eigenvalue equation to calculate the KRIC and solve the problem. The computational cost for parameter tuning by the KRIC is reduced drastically by using the Nystro/spl uml/m approximation. The test error rate of SVMs or KLR with the regularization parameter tuned by the KRIC is comparable with the one by the cross validation or evaluation of the evidence. The computational cost of the KRIC is significantly lower than the one of the other criteria.     

Feature selection using support vector machines and bootstrap methods for ventricular fibrillation detection

Early detection of ventricular fibrillation (VF) is crucial for the success of the defibrillation therapy in automatic devices. A high number of detectors have been proposed based on temporal, spectral, and time-frequency parameters extracted from the surface electrocardiogram (ECG), showing always a limited performance. The combination ECG parameters on different domain (time, frequency, and time-frequency) using machine learning algorithms has been used to improve detection efficiency. However, the potential utilization of a wide number of parameters benefiting machine learning schemes has raised the need of efficient feature selection (FS) procedures. In this study, we propose a novel FS algorithm based on support vector machines (SVM) classifiers and bootstrap resampling (BR) techniques. We define a backward FS procedure that relies on evaluating changes in SVM performance when removing features from the input space. This evaluation is achieved according to a nonparametric statistic based on BR. After simulation studies, we benchmark the performance of our FS algorithm in AHA and MIT-BIH ECG databases. Our results show that the proposed FS algorithm outperforms the recursive feature elimination method in synthetic examples, and that the VF detector performance improves with the reduced feature set.     

级联SVM和分类器融合的人脸性别识别方法

传统的支持向量机（Support Vector Machines,SVM）在面对大样本训练问题时,其样本数量会受到内存的限制。因此,提出一种基于级联SVM和分类器融合的人脸图像性别识别方法。级联SVM分类器可以通过设定阈值将识别难易程度不同的样本分成若干层次来进行训练;同时,在级联的每一层上,为了降低分类器在识别过程中受各种因素的影响,对不同特征维数下得到的最优分类器进行融合,通过融合减小误差,使中性的人脸样本有更明确的分类。在同一硬件条件下的实验结果表明,单层SVM最多只能训练7万样本,而四层级联SVM训练样本数可达12万以上,相应的识别率也从单层融合前的96.7%上升至四层融合后的99.1%。     

支持向量机和最小二乘支持向量机的比较及应用研究

介绍和比较了支持向量机分类器和量小二乘支持向量机分类器的算法。并将支持向量机分类器和量小二乘支持向量机分类器应用于心脏病诊断，取得了较高的准确率。所用数据来自UCI bench—mark数据集。实验结果表明，支持向量机和量小二乘支持向量机在医疗诊断中有很大的应用潜力。