SVM在多源遥感图像分类中的应用研究

在利用遥感图像进行土地利用/覆盖分类过程中,可采用以下两种途径来提高分类精度:一是通过增加有利于分类的数据源,引入地理辅助数据和归一化植被指数(NDVI)来进行多源信息融合;二是选择更好的分类方法,例如支持向量机(SVM)学习方法,由于该方法克服了最大似然法和神经网络的弱点,非常适合高维、复杂的小样本多源数据的分类。为了提高多源遥感图像分类的精度,还研究了支持向量机在遥感图像分类中模型的选择,包括多类模型和核函数的选择。分类结果表明,支持向量机比传统的分类方法具有更高的精度,尤其是基于径向基核函数和一对一多类方法的支持向量机模型更适合多源遥感图像分类,因此,基于支持向量机的多源土地利用/覆盖分类能大大提高分类精度。     

On classification of signals represented with data-dependent overcomplete dictionaries

This paper focuses on the problem of how data representation influences the generalization error of kernel-based learning machines like support vector machines (SVMs). We analyse the effects of sparse and dense data representations on the generalization error of SVM. We show that using sparse representations the performances of classifiers belonging to hypothesis spaces induced by polynomial or Gaussian kernel functions reduce to the performances of linear classifiers. Sparse representations reduce the generalization error as long as the representation is not too sparse as with very large dictionaries. Dense data representations reduce the generalization error also using very large dictionaries. We use two schemes for representing data in data-independent overcomplete Haar and Gabor dictionaries, and measure the generalization error of SVMs on benchmark datasets. We study sparse and dense representations in the case of data-dependent overcomplete dictionaries and we show how this leads to principal component analysis.     

一种新型的多元分类支持向量机

最小二乘支持向量机采用最小二乘线性系统代替传统的支持向量机采用二次规划方法解决模式识别问题。该文详细推理和分析了二元分类最小二乘支持向量机算法，构建了多元分类最小二乘支持向量机，并通过典型样本进行测试，结果表明采用多元分类最小二乘支持向量机进行模式识别是有效、可行的。     

最小二乘支持向量机算法研究

1 引言支持向量机(SVM,Support Vector Machines)是基于结构风险最小化的统计学习方法,它具有完备的统计学习理论基础和出色的学习性能,在模式识别和函数估计中得到了有效的应用(Vapnik,1995,1998)。支持向量机方法一方面通过把数据映射到高维空间,解决原始空间中数据线性不可分问题;另一方面,通过构造最优分类超平面进行数据分类。神经网络通过基于梯度迭代的方法进行数据学习,容易陷入局部最小值,支持向量机是通过解决一个二次规划问题,来获得     

多支持向量机在线联合的运动目标跟踪算法

依据二元分类的思想,提出了一种新的基于多支持向量机在线联合的运动目标跟踪算法。首先选择线性支持向量机作为分类器最大限度地将目标和背景区分开来,对线性支持向量机进行简单高效的在线更新,采用支持向量自动记录运动目标 “关键帧”的信息。然后通过Adaboost算法为每个线性支持向量机分别赋以不同的权重,进行在线联合获得强分类器。实验结果表明,该算法具有较强的鲁棒性,尤其在目标变化过于激烈的情况下能够实现较为稳定的跟踪。     

Comparison of artificial neural networks and support vector machine classifiers for land cover classification in Northern China using a SPOT-5 HRG image

This article presents a sufficient comparison of two types of advanced non-parametric classifiers implemented in remote sensing for land cover classification. A SPOT-5 HRG image of Yanqing County, Beijing, China, was used, in which agriculture and forest dominate land use. Artificial neural networks (ANNs), including the adaptive backpropagation (ABP) algorithm, Levenberg–Marquardt (LM) algorithm, Quasi-Newton (QN) algorithm and radial basis function (RBF) were carefully tested. The LM–ANN and RBF–ANN, which outperform the other two, were selected to make a detailed comparison with support vector machines (SVMs). The experiments show that those well-trained ANNs and SVMs have no significant difference in classification accuracy, but the SVM usually performs slightly better. Analysis of the effect of the training set size highlights that the SVM classifier has great tolerance on a small training set and avoids the problem of insufficient training of ANN classifiers. The testing also illustrates that the ANNs and SVMs can vary greatly with regard to training time. The LM–ANN can converge very quickly but not in a stable manner. By contrast, the training of RBF–ANN and SVM classifiers is fast and can be repeatable.     

A semi-dependent decomposition approach to learn hierarchical classifiers

In hierarchical classification, classes are arranged in a hierarchy represented by a tree or a forest, and each example is labeled with a set of classes located on paths from roots to leaves or internal nodes. In other words, both multiple and partial paths are allowed. A straightforward approach to learn a hierarchical classifier, usually used as a baseline method, consists in learning one binary classifier for each node of the hierarchy; the hierarchical classifier is then obtained using a top-down evaluation procedure. The main drawback of this naive approach is that these binary classifiers are constructed independently, when it is clear that there are dependencies between them that are motivated by the hierarchy and the evaluation procedure employed. In this paper, we present a new decomposition method in which each node classifier is built taking into account other classifiers, its descendants, and the loss function used to measure the goodness of hierarchical classifiers. Following a bottom-up learning strategy, the idea is to optimize the loss function at every subtree assuming that all classifiers are known except the one at the root. Experimental results show that the proposed approach has accuracies comparable to state-of-the-art hierarchical algorithms and is better than the naive baseline method described above. Moreover, the benefits of our proposal include the possibility of parallel implementations, as well as the use of all available well-known techniques to tune binary classification SVMs.     

Using support vector machines and acoustic noise signal for degradation analysis of rotating machinery

An automated approach to degradation analysis is proposed that uses a rotating machine’s acoustic signal to determine Remaining Useful Life (RUL). High resolution spectral features are extracted from the acoustic data collected over the entire lifetime of the machine. A novel approach to the computation of Mutual Information based Feature Subset Selection is applied, to remove redundant and irrelevant features, that does not require class label boundaries of the dataset or spectral locations of developing defect to be known or pre-estimated. Using subsets of the feature space, multi-class linear and Radial Basis Function (RBF) Support Vector Machine (SVM) classifiers are developed and a comparison of their performance is provided. Performance of all classifiers is found to be very high, 85 to 98%, with RBF SVMs outperforming linear SVMs when a smaller number of features are used. As larger numbers of features are used for classification, the problem space becomes more linearly separable and the linear SVMs are shown to have comparable performance. A detailed analysis of the misclassifications is provided and an approach to better understand and interpret costly misclassifications is discussed. While defining class label boundaries using an automated k-means clustering algorithm improves performance with an accuracy of approximately 99%, further analysis shows that in 88% of all misclassifications the actual class of failure had the next highest probability of occurring. Thus, a system that incorporates probability distributions as a measure of confidence for the predicted RUL would provide additional valuable information for scheduling preventative maintenance. This work was supported by IDA Ireland.     

IP-LSSVM: A two-step sparse classifier

We present in this work a two-step sparse classifier called IP-LSSVM which is based on Least Squares Support Vector Machine (LS-SVM). The formulation of LS-SVM aims at solving the learning problem with a system of linear equations. Although this solution is simpler, there is a loss of sparseness in the feature vectors. Many works on LS-SVM are focused on improving support vectors representation in the least squares approach, since they correspond to the only vectors that must be stored for further usage of the machine, which can also be directly used as a reduced subset that represents the initial one. The proposed classifier incorporates the advantages of either SVM and LS-SVM: automatic detection of support vectors and a solution obtained simply by the solution of systems of linear equations. IP-LSSVM was compared with other sparse LS-SVM classifiers from literature, and RRS+LS-SVM. The experiments were performed on four important benchmark databases in Machine Learning and on two artificial databases created to show visually the support vectors detected. The results show that IP-LSSVM represents a viable alternative to SVMs, since both have similar features, supported by literature results and yet IP-LSSVM has a simpler and more understandable formulation.     

A hybrid SVM based decision tree

We have proposed a hybrid SVM based decision tree to speedup SVMs in its testing phase for binary classification tasks. While most existing methods addressed towards this task aim at reducing the number of support vectors, we have focused on reducing the number of test datapoints that need SVM’s help in getting classified. The central idea is to approximate the decision boundary of SVM using decision trees. The resulting tree is a hybrid tree in the sense that it has both univariate and multivariate (SVM) nodes. The hybrid tree takes SVM’s help only in classifying crucial datapoints lying near decision boundary; remaining less crucial datapoints are classified by fast univariate nodes. The classification accuracy of the hybrid tree is guaranteed by tuning a threshold parameter. Extensive computational comparisons on 19 publicly available datasets indicate that the proposed method achieves significant speedup when compared to SVMs, without any compromise in classification accuracy.     

Financial decision support using neural networks and support vector machines

Abstract: Bankruptcy prediction and credit scoring are the two important problems facing financial decision support. The multilayer perceptron (MLP) network has shown its applicability to these problems and its performance is usually superior to those of other traditional statistical models. Support vector machines (SVMs) are the core machine learning techniques and have been used to compare with MLP as the benchmark. However, the performance of SVMs is not fully understood in the literature because an insufficient number of data sets is considered and different kernel functions are used to train the SVMs. In this paper, four public data sets are used. In particular, three different sizes of training and testing data in each of the four data sets are considered (i.e. 3:7, 1:1 and 7:3) in order to examine and fully understand the performance of SVMs. For SVM model construction, the linear, radial basis function and polynomial kernel functions are used to construct the SVMs. Using MLP as the benchmark, the SVM classifier only performs better in one of the four data sets. On the other hand, the prediction results of the MLP and SVM classifiers are not significantly different for the three different sizes of training and testing data.     

Linear models for cost‐sensitive classification

In this paper, we investigate the performance of statistical, mathematical programming and heuristic linear models for cost‐sensitive classification. In particular, we use five cost‐sensitive techniques including Fisher's discriminant analysis (DA), asymmetric misclassification cost mixed integer programming (AMC‐MIP), cost‐sensitive support vector machine (CS‐SVM), a hybrid support vector machine and mixed integer programming (SVMIP) and heuristic cost‐sensitive genetic algorithm (CGA) techniques. Using simulated datasets of varying group overlaps, data distributions and class biases, and real‐world datasets from financial and medical domains, we compare the performances of our five techniques based on overall holdout sample misclassification cost. The results of our experiments on simulated datasets indicate that when group overlap is low and data distribution is exponential, DA appears to provide superior performance. For all other situations with simulated datasets, CS‐SVM provides superior performance. In case of real‐world datasets from financial domain, CGA and AMC‐MIP hold a slight edge over the two SVM‐based classifiers. However, for medical domains with mixed continuous and discrete attributes, SVM classifiers perform better than heuristic (CGA) and AMC‐MIP classifiers. The SVMIP model is the most computationally inefficient model and poor performing model.     

基于SVM 的二叉树多类分类算法及其在故障诊断中的应用

基于结构风险最小化原则的支持向量机(SVM)对小样本决策具有较好的学习推广性。但由于常规SVM算法是从2类分类问题推导出的，在解决故障诊断这种典型的多类分类问题时存在因雄，因而提出一种依赖故障优先级的基于SVM的二叉树多级分类器实现(2PTMC)方法，该方法具有简单、直观，重复训练样本少的优点。通过将其应用于柴油机振动信号的故障诊断，获得了令人满意的效果。     

Linear classifier design under heteroscedasticity in Linear Discriminant Analysis

Under normality and homoscedasticity assumptions, Linear Discriminant Analysis (LDA) is known to be optimal in terms of minimising the Bayes error for binary classification. In the heteroscedastic case, LDA is not guaranteed to minimise this error. Assuming heteroscedasticity, we derive a linear classifier, the Gaussian Linear Discriminant (GLD), that directly minimises the Bayes error for binary classification. In addition, we also propose a local neighbourhood search (LNS) algorithm to obtain a more robust classifier if the data is known to have a non-normal distribution. We evaluate the proposed classifiers on two artificial and ten real-world datasets that cut across a wide range of application areas including handwriting recognition, medical diagnosis and remote sensing, and then compare our algorithm against existing LDA approaches and other linear classifiers. The GLD is shown to outperform the original LDA procedure in terms of the classification accuracy under heteroscedasticity. While it compares favourably with other existing heteroscedastic LDA approaches, the GLD requires as much as 60 times lower training time on some datasets. Our comparison with the support vector machine (SVM) also shows that, the GLD, together with the LNS, requires as much as 150 times lower training time to achieve an equivalent classification accuracy on some of the datasets. Thus, our algorithms can provide a cheap and reliable option for classification in a lot of expert systems.     

Type-2 fuzzy logic-based classifier fusion for support vector machines

As a machine-learning tool, support vector machines (SVMs) have been gaining popularity due to their promising performance. However, the generalization abilities of SVMs often rely on whether the selected kernel functions are suitable for real classification data. To lessen the sensitivity of different kernels in SVMs classification and improve SVMs generalization ability, this paper proposes a fuzzy fusion model to combine multiple SVMs classifiers. To better handle uncertainties existing in real classification data and in the membership functions (MFs) in the traditional type-1 fuzzy logic system (FLS), we apply interval type-2 fuzzy sets to construct a type-2 SVMs fusion FLS. This type-2 fusion architecture takes considerations of the classification results from individual SVMs classifiers and generates the combined classification decision as the output. Besides the distances of data examples to SVMs hyperplanes, the type-2 fuzzy SVMs fusion system also considers the accuracy information of individual SVMs. Our experiments show that the type-2 based SVM fusion classifiers outperform individual SVM classifiers in most cases. The experiments also show that the type-2 fuzzy logic-based SVMs fusion model is better than the type-1 based SVM fusion model in general.     

Color image watermark extraction based on support vector machines

This work proposes a novel watermarking technique called SVM-based Color Image Watermarking (SCIW), based on support vector machines (SVMs) for the authentication of color images. To protect the copyright of a color image, a signature (a watermark), which is represented by a sequence of binary data, is embedded in the color image. The watermark-extraction issue can be treated as a classification problem involving binary classes. The SCIW method constructs a set of training patterns with the use of binary labels by employing three image features, which are the differences between a local image statistic and the luminance value of the center pixel in a sliding window with three distinct shapes. This set of training patterns is gathered from a pair of images, an original image and its corresponding watermarked image in the spatial domain. A quasi-optimal hyperplane (a binary classifier) can be realized by an SVM. The SCIW method utilizes this set of training patterns to train the SVM and then applies the trained SVM to classify a set of testing patterns. Following the results produced by the classifier (the trained SVM), the SCIW method retrieves the hidden signature without the original image during watermark extraction. Experimental results have demonstrated that the SCIW method is sufficiently robust against several color-image manipulations, and that it outperforms other proposed methods considered in this work.     

Rough set based 1-v-1 and 1-v-r approaches to support vector machine multi-classification

Support vector machines (SVMs) are essentially binary classifiers. To improve their applicability, several methods have been suggested for extending SVMs for multi-classification, including one-versus-one (1-v-1), one-versus-rest (1-v-r) and DAGSVM. In this paper, we first describe how binary classification with SVMs can be interpreted using rough sets. A rough set approach to SVM classification removes the necessity of exact classification and is especially useful when dealing with noisy data. Next, by utilizing the boundary region in rough sets, we suggest two new approaches, extensions of 1-v-r and 1-v-1, to SVM multi-classification that allow for an error rate. We explicitly demonstrate how our extended 1-v-r may shorten the training time of the conventional 1-v-r approach. In addition, we show that our 1-v-1 approach may have reduced storage requirements compared to the conventional 1-v-1 and DAGSVM techniques. Our techniques also provide better semantic interpretations of the classification process. The theoretical conclusions are supported by experimental findings involving a synthetic dataset.     

Classification of acoustic events using SVM-based clustering schemes

Acoustic events produced in controlled environments may carry information useful for perceptually aware interfaces. In this paper we focus on the problem of classifying 16 types of meeting-room acoustic events. First of all, we have defined the events and gathered a sound database. Then, several classifiers based on support vector machines (SVM) are developed using confusion matrix based clustering schemes to deal with the multi-class problem. Also, several sets of acoustic features are defined and used in the classification tests. In the experiments, the developed SVM-based classifiers are compared with an already reported binary tree scheme and with their correlative Gaussian mixture model (GMM) classifiers. The best results are obtained with a tree SVM-based classifier that may use a different feature set at each node. With it, a 31.5% relative average error reduction is obtained with respect to the best result from a conventional binary tree scheme.     

基于基因表达谱的SRBCT分类研究

肿瘤亚型的准确判别对肿瘤的治疗具有重要的意义。文章提出了一种多类肿瘤分类和特征基因选取的策略。该方法以儿童SRBCT(小圆蓝细胞瘤)的基因表达谱为例,计算基因的类加权Bhattacharyya距离,并据此选取特征基因,然后利用这些基因的表达数据建立了基于支持向量机的多模预测模型并对SRBCT的4种亚型进行了识别。实验结果表明了该方法的有效性和可行性。     

Classifier ensemble construction with rotation forest to improve medical diagnosis performance of machine learning algorithms

Improving accuracies of machine learning algorithms is vital in designing high performance computer-aided diagnosis (CADx) systems. Researches have shown that a base classifier performance might be enhanced by ensemble classification strategies. In this study, we construct rotation forest (RF) ensemble classifiers of 30 machine learning algorithms to evaluate their classification performances using Parkinson's, diabetes and heart diseases from literature.While making experiments, first the feature dimension of three datasets is reduced using correlation based feature selection (CFS) algorithm. Second, classification performances of 30 machine learning algorithms are calculated for three datasets. Third, 30 classifier ensembles are constructed based on RF algorithm to assess performances of respective classifiers with the same disease data. All the experiments are carried out with leave-one-out validation strategy and the performances of the 60 algorithms are evaluated using three metrics; classification accuracy (ACC), kappa error (KE) and area under the receiver operating characteristic (ROC) curve (AUC).Base classifiers succeeded 72.15%, 77.52% and 84.43% average accuracies for diabetes, heart and Parkinson's datasets, respectively. As for RF classifier ensembles, they produced average accuracies of 74.47%, 80.49% and 87.13% for respective diseases.RF, a newly proposed classifier ensemble algorithm, might be used to improve accuracy of miscellaneous machine learning algorithms to design advanced CADx systems.