SVM在多源遥感图像分类中的应用研究

在利用遥感图像进行土地利用/覆盖分类过程中,可采用以下两种途径来提高分类精度:一是通过增加有利于分类的数据源,引入地理辅助数据和归一化植被指数(NDVI)来进行多源信息融合;二是选择更好的分类方法,例如支持向量机(SVM)学习方法,由于该方法克服了最大似然法和神经网络的弱点,非常适合高维、复杂的小样本多源数据的分类。为了提高多源遥感图像分类的精度,还研究了支持向量机在遥感图像分类中模型的选择,包括多类模型和核函数的选择。分类结果表明,支持向量机比传统的分类方法具有更高的精度,尤其是基于径向基核函数和一对一多类方法的支持向量机模型更适合多源遥感图像分类,因此,基于支持向量机的多源土地利用/覆盖分类能大大提高分类精度。     

Multispectral landuse classification using neural networks and support vector machines: one or the other,or both?

Land use classification is an important part of many remote sensing applications. A lot of research has gone into the application of statistical and neural network classifiers to remote‐sensing images. This research involves the study and implementation of a new pattern recognition technique introduced within the framework of statistical learning theory called Support Vector Machines (SVMs), and its application to remote‐sensing image classification. Standard classifiers such as Artificial Neural Network (ANN) need a number of training samples that exponentially increase with the dimension of the input feature space. With a limited number of training samples, the classification rate thus decreases as the dimensionality increases. SVMs are independent of the dimensionality of feature space as the main idea behind this classification technique is to separate the classes with a surface that maximizes the margin between them, using boundary pixels to create the decision surface. Results from SVMs are compared with traditional Maximum Likelihood Classification (MLC) and an ANN classifier. The findings suggest that the ANN and SVM classifiers perform better than the traditional MLC. The SVM and the ANN show comparable results. However, accuracy is dependent on factors such as the number of hidden nodes (in the case of ANN) and kernel parameters (in the case of SVM). The training time taken by the SVM is several magnitudes less.     

Benchmarking Least Squares Support Vector Machine Classifiers

In Support Vector Machines (SVMs), the solution of the classification problem is characterized by a (convex) quadratic programming (QP) problem. In a modified version of SVMs, called Least Squares SVM classifiers (LS-SVMs), a least squares cost function is proposed so as to obtain a linear set of equations in the dual space. While the SVM classifier has a large margin interpretation, the LS-SVM formulation is related in this paper to a ridge regression approach for classification with binary targets and to Fisher's linear discriminant analysis in the feature space. Multiclass categorization problems are represented by a set of binary classifiers using different output coding schemes. While regularization is used to control the effective number of parameters of the LS-SVM classifier, the sparseness property of SVMs is lost due to the choice of the 2-norm. Sparseness can be imposed in a second stage by gradually pruning the support value spectrum and optimizing the hyperparameters during the sparse approximation procedure. In this paper, twenty public domain benchmark datasets are used to evaluate the test set performance of LS-SVM classifiers with linear, polynomial and radial basis function (RBF) kernels. Both the SVM and LS-SVM classifier with RBF kernel in combination with standard cross-validation procedures for hyperparameter selection achieve comparable test set performances. These SVM and LS-SVM performances are consistently very good when compared to a variety of methods described in the literature including decision tree based algorithms, statistical algorithms and instance based learning methods. We show on ten UCI datasets that the LS-SVM sparse approximation procedure can be successfully applied.     

Support Vector Machines for TCP traffic classification

Support Vector Machines (SVM) represent one of the most promising Machine Learning (ML) tools that can be applied to the problem of traffic classification in IP networks. In the case of SVMs, there are still open questions that need to be addressed before they can be generally applied to traffic classifiers. Having being designed essentially as techniques for binary classification, their generalization to multi-class problems is still under research. Furthermore, their performance is highly susceptible to the correct optimization of their working parameters. In this paper we describe an approach to traffic classification based on SVM. We apply one of the approaches to solving multi-class problems with SVMs to the task of statistical traffic classification, and describe a simple optimization algorithm that allows the classifier to perform correctly with as little training as a few hundred samples. The accuracy of the proposed classifier is then evaluated over three sets of traffic traces, coming from different topological points in the Internet. Although the results are relatively preliminary, they confirm that SVM-based classifiers can be very effective at discriminating traffic generated by different applications, even with reduced training set sizes.     

An assessment of the effectiveness of decision tree methods for land cover classification

Choice of a classification algorithm is generally based upon a number of factors, among which are availability of software, ease of use, and performance, measured here by overall classification accuracy. The maximum likelihood (ML) procedure is, for many users, the algorithm of choice because of its ready availability and the fact that it does not require an extended training process. Artificial neural networks (ANNs) are now widely used by researchers, but their operational applications are hindered by the need for the user to specify the configuration of the network architecture and to provide values for a number of parameters, both of which affect performance. The ANN also requires an extended training phase.In the past few years, the use of decision trees (DTs) to classify remotely sensed data has increased. Proponents of the method claim that it has a number of advantages over the ML and ANN algorithms. The DT is computationally fast, make no statistical assumptions, and can handle data that are represented on different measurement scales. Software to implement DTs is readily available over the Internet. Pruning of DTs can make them smaller and more easily interpretable, while the use of boosting techniques can improve performance.In this study, separate test and training data sets from two different geographical areas and two different sensors—multispectral Landsat ETM+ and hyperspectral DAIS—are used to evaluate the performance of univariate and multivariate DTs for land cover classification. Factors considered are: the effects of variations in training data set size and of the dimensionality of the feature space, together with the impact of boosting, attribute selection measures, and pruning. The level of classification accuracy achieved by the DT is compared to results from back-propagating ANN and the ML classifiers. Our results indicate that the performance of the univariate DT is acceptably good in comparison with that of other classifiers, except with high-dimensional data. Classification accuracy increases linearly with training data set size to a limit of 300 pixels per class in this case. Multivariate DTs do not appear to perform better than univariate DTs. While boosting produces an increase in classification accuracy of between 3% and 6%, the use of attribute selection methods does not appear to be justified in terms of accuracy increases. However, neither the univariate DT nor the multivariate DT performed as well as the ANN or ML classifiers with high-dimensional data.     

一种基于个人身份认证的正面人脸识别算法

利用小波分解提取人脸特征技术和支持向量机 (SVM)分类模型 ,提出了一种基于个人身份认证的正面人脸识别算法 (或称为人脸认证方法 ) .针对 M个用户的人脸认证算法包括二个阶段 :(1)训练阶段 :使用小波分解方法对脸像训练集中的人脸图象进行特征提取 ,并用所提取的人脸特征向量训练 M个 SVM(对应 M个用户 ) ;(2 )认证阶段 :先由待认证者所声称的用户身份 (姓名或密码等 )确定对应的一训练好的 SVM,然后用这一 SVM对小波分解方法提取的待认证人的脸像特征向量进行分类 ,分类结果将显示待认证人所声称的身份是否真实 .利用 ORL人脸图象库对该算法的实验测试结果 ,以及与径向基函数神经网络作为分类器时的实验结果比较表明了该算法性能的优越性     

Fast and accurate text classification via multiple linear discriminant projections

Support vector machines (SVMs) have shown superb performance for text classification tasks. They are accurate, robust, and quick to apply to test instances. Their only potential drawback is their training time and memory requirement. For n training instances held in memory, the best-known SVM implementations take time proportional to n^a, where a is typically between 1.8 and 2.1. SVMs have been trained on data sets with several thousand instances, but Web directories today contain millions of instances that are valuable for mapping billions of Web pages into Yahoo!-like directories. We present SIMPL, a nearly linear-time classification algorithm that mimics the strengths of SVMs while avoiding the training bottleneck. It uses Fisher's linear discriminant, a classical tool from statistical pattern recognition, to project training instances to a carefully selected low-dimensional subspace before inducing a decision tree on the projected instances. SIMPL uses efficient sequential scans and sorts and is comparable in speed and memory scalability to widely used naive Bayes (NB) classifiers, but it beats NB accuracy decisively. It not only approaches and sometimes exceeds SVM accuracy, but also beats the running time of a popular SVM implementation by orders of magnitude. While describing SIMPL, we make a detailed experimental comparison of SVM-generated discriminants with Fisher's discriminants, and we also report on an analysis of the cache performance of a popular SVM implementation. Our analysis shows that SIMPL has the potential to be the method of choice for practitioners who want the accuracy of SVMs and the simplicity and speed of naive Bayes classifiers.Received: 9 September 2002, Accepted: 3 March 2003, Published online: 21 July 2003 Edited by Y. Ioannidis     

A novel pattern recognition algorithm: Combining ART network with SVM to reconstruct a multi-class classifier

Based on the principle of one-against-one support vector machines (SVMs) multi-class classification algorithm, this paper proposes an extended SVMs method which couples adaptive resonance theory (ART) network to reconstruct a multi-class classifier. Different coupling strategies to reconstruct a multi-class classifier from binary SVM classifiers are compared with application to fault diagnosis of transmission line. Majority voting, a mixture matrix and self-organizing map (SOM) network are compared in reconstructing the global classification decision. In order to evaluate the method’s efficiency, one-against-all, decision directed acyclic graph (DDAG) and decision-tree (DT) algorithm based SVM are compared too. The comparison is done with simulations and the best method is validated with experimental data.     

Use of particle swarm optimization for machinery fault detection

A study is presented on the application of particle swarm optimization (PSO) combined with other computational intelligence (CI) techniques for bearing fault detection in machines. The performance of two CI based classifiers, namely, artificial neural networks (ANNs) and support vector machines (SVMs) are compared. The time domain vibration signals of a rotating machine with normal and defective bearings are processed for feature extraction. The extracted features from original and preprocessed signals are used as inputs to the classifiers for detection of machine condition. The classifier parameters, e.g., the number of nodes in the hidden layer for ANNs and the kernel parameters for SVMs are selected along with input features using PSO algorithms. The classifiers are trained with a subset of the experimental data for known machine conditions and are tested using the remaining set of data. The procedure is illustrated using the experimental vibration data of a rotating machine. The roles of the number of features, PSO parameters and CI classifiers on the detection success are investigated. Results are compared with other techniques such as genetic algorithm (GA) and principal component analysis (PCA). The PSO based approach gave a test classification success rate of 98.6–100% which were comparable with GA and much better than with PCA. The results show the effectiveness of the selected features and the classifiers in the detection of the machine condition.     

Image mining by spectral features: A case study of scenery image classification

Spectral features of images, such as Gabor filters and wavelet transform can be used for texture image classification. That is, a classifier is trained based on some labeled texture features as the training set to classify unlabeled texture features of images into some pre-defined classes. The aim of this paper is twofold. First, it investigates the classification performance of using Gabor filters, wavelet transform, and their combination respectively, as the texture feature representation of scenery images (such as mountain, castle, etc.). A k-nearest neighbor (k-NN) classifier and support vector machine (SVM) are also compared. Second, three k-NN classifiers and three SVMs are combined respectively, in which each of the combined three classifiers uses one of the above three texture feature representations respectively, to see whether combining multiple classifiers can outperform the single classifier in terms of scenery image classification. The result shows that a single SVM using Gabor filters provides the highest classification accuracy than the other two spectral features and the combined three k-NN classifiers and three SVMs.     

Artificial neural networks and genetic algorithm for bearing fault detection

A study is presented to compare the performance of three types of artificial neural network (ANN), namely, multi layer perceptron (MLP), radial basis function (RBF) network and probabilistic neural network (PNN), for bearing fault detection. Features are extracted from time domain vibration signals, without and with preprocessing, of a rotating machine with normal and defective bearings. The extracted features are used as inputs to all three ANN classifiers: MLP, RBF and PNN for two- class (normal or fault) recognition. Genetic algorithms (GAs) have been used to select the characteristic parameters of the classifiers and the input features. For each trial, the ANNs are trained with a subset of the experimental data for known machine conditions. The ANNs are tested using the remaining set of data. The procedure is illustrated using the experimental vibration data of a rotating machine. The roles of different vibration signals and preprocessing techniques are investigated. The results show the effectiveness of the features and the classifiers in detection of machine condition.     

Type-2 fuzzy logic-based classifier fusion for support vector machines

As a machine-learning tool, support vector machines (SVMs) have been gaining popularity due to their promising performance. However, the generalization abilities of SVMs often rely on whether the selected kernel functions are suitable for real classification data. To lessen the sensitivity of different kernels in SVMs classification and improve SVMs generalization ability, this paper proposes a fuzzy fusion model to combine multiple SVMs classifiers. To better handle uncertainties existing in real classification data and in the membership functions (MFs) in the traditional type-1 fuzzy logic system (FLS), we apply interval type-2 fuzzy sets to construct a type-2 SVMs fusion FLS. This type-2 fusion architecture takes considerations of the classification results from individual SVMs classifiers and generates the combined classification decision as the output. Besides the distances of data examples to SVMs hyperplanes, the type-2 fuzzy SVMs fusion system also considers the accuracy information of individual SVMs. Our experiments show that the type-2 based SVM fusion classifiers outperform individual SVM classifiers in most cases. The experiments also show that the type-2 fuzzy logic-based SVMs fusion model is better than the type-1 based SVM fusion model in general.     

一种基于混合二叉树结构的多类支持向量机分类算法

为提高多类支持向量机的分类效率,提出了一种基于混合二叉树结构的多类支持向量机分类算法。该混合二叉树中的每个内部结点对应一个分割超平面,该超平面通过计算两个距离最远的类的质心而获得,即该超平面为连接两质心线段的垂直平分线。每个终端结点(即决策结点)对应一个支持向量机,它的训练集不再是质心而是两类(组)样本集。该分类模型通常是超平面和支持向量机的混合结构,其中超平面实现训练早期的近似划分,以提升分类速度;而支持向量机完成最终的精确分类,以保证分类精度。实验结果表明,相比于经典的多类支持向量机方法,该算法在保证分类精度的前提下,能够有效缩短计算时间,提升分类效率。     

Improving the SVM gender classification accuracy using clustering and incremental learning

Gender recognition has been playing a very important role in various applications such as human–computer interaction, surveillance, and security. Nonlinear support vector machines (SVMs) were investigated for the identification of gender using the Face Recognition Technology (FERET) image face database. It was shown that SVM classifiers outperform the traditional pattern classifiers (linear, quadratic, Fisher linear discriminant, and nearest neighbour). In this context, this paper aims to improve the SVM classification accuracy in the gender classification system and propose new models for a better performance. We have evaluated different SVM learning algorithms; the SVM‐radial basis function with a 5% outlier fraction outperformed other SVM classifiers. We have examined the effectiveness of different feature selection methods. AdaBoost performs better than the other feature selection methods in selecting the most discriminating features. We have proposed two classification methods that focus on training subsets of images among the training images. Method 1 combines the outcome of different classifiers based on different image subsets, whereas method 2 is based on clustering the training data and building a classifier for each cluster. Experimental results showed that both methods have increased the classification accuracy.     

一种LDA与SVM混合的多类分类方法

针对决策有向无环图支持向量机（ＤＤＡＧＳＶＭ）需训练大量支持向量机（ＳＶＭ）和误差积累的问题,提出一种线性判别分析（ＬＤＡ）与ＳＶＭ 混合的多类分类算法．首先根据高维样本在低维空间中投影的特点,给出一种优化ＬＤＡ 分类阈值;然后以优化ＬＤＡ 对每个二类问题的分类误差作为类间线性可分度,对线性可分度较低的问题采用非线性ＳＶＭ 加以解决,并以分类误差作为对应二类问题的可分度;最后将可分度作为混合ＤＤＡＧ 分类器的决策依据．实验表明,与ＤＤＡＧＳＶＭ 相比,所提出算法在确保泛化精度的条件下具有更高的训练和分类速度．     

Editorial

This study investigates the potential of applying the radial basis function (RBF) neural network architecture for the classification of multispectral very high spatial resolution satellite images into 13 classes of various scales. For the development of the RBF classifiers, the innovative fuzzy means training algorithm is utilized, which is based on a fuzzy partition of the input space. The method requires only a short amount of time to select both the structure and the parameters of the RBF classifier. The new technique was applied to the area of Lake Kerkini, which is a wetland of great ecological value, located in northern Greece. Eleven experiments were carried out in total in order to investigate the performance of the classifier using different input parameters (spectral and textural) as well as different window sizes and neural network complexities. For comparison purposes the same satellite scene was classified using the maximum likelihood (MLH) classification with the same set of training samples. Overall, the neural network classifiers outperformed the MLH classification by 10–17%, reaching a maximum overall accuracy of 78%. Analysis showed that the selection of input parameters is vital for the success of the classifiers. On the other hand, the incorporation of textural analysis and/or modification of the window size do not affect the performance substantially.     

基于元学习和叠加法的双层支持向量机算法

提出一种模式识别算法——双层支持量机算法,用来提高表面肌电识别精度。该算法融合集成学习中元学习的并行方法和叠加法的递进思想,把基本SVM分类器并行分布在第1层,第1层的预测结果作为第2层的输入,由第2层再进行分类识别,从而通过多层分类器组合来融合多源特征。以手臂表面肌电数据集为测试数据,采用文中的双层支持向量机,各肌肉的肌电信号分别输入基支持向量机,组合器融合各肌肉电信号特征,集成识别前臂肌肉群的肌电信号,从而实现运动意图的精确识别。实验结果显示,在预测精度上,此算法优于单个SVM分类器。在预测性能上(识别精度、耗时、鲁棒性),此算法优于随机森林和旋转森林等集成分类器。     

Using support vector machines and acoustic noise signal for degradation analysis of rotating machinery

An automated approach to degradation analysis is proposed that uses a rotating machine’s acoustic signal to determine Remaining Useful Life (RUL). High resolution spectral features are extracted from the acoustic data collected over the entire lifetime of the machine. A novel approach to the computation of Mutual Information based Feature Subset Selection is applied, to remove redundant and irrelevant features, that does not require class label boundaries of the dataset or spectral locations of developing defect to be known or pre-estimated. Using subsets of the feature space, multi-class linear and Radial Basis Function (RBF) Support Vector Machine (SVM) classifiers are developed and a comparison of their performance is provided. Performance of all classifiers is found to be very high, 85 to 98%, with RBF SVMs outperforming linear SVMs when a smaller number of features are used. As larger numbers of features are used for classification, the problem space becomes more linearly separable and the linear SVMs are shown to have comparable performance. A detailed analysis of the misclassifications is provided and an approach to better understand and interpret costly misclassifications is discussed. While defining class label boundaries using an automated k-means clustering algorithm improves performance with an accuracy of approximately 99%, further analysis shows that in 88% of all misclassifications the actual class of failure had the next highest probability of occurring. Thus, a system that incorporates probability distributions as a measure of confidence for the predicted RUL would provide additional valuable information for scheduling preventative maintenance. This work was supported by IDA Ireland.     

Improvement of land-cover classification over frequently cloud-covered areas using Landsat 8 time-series composites and an ensemble of supervised classifiers

Recent abundance of moderate-to-high spatial resolution satellite imagery has facilitated land-cover map production. However, in cloud-prone areas, building high-resolution land-cover maps is still challenging due to infrequent satellite revisits and lack of cloud-free data. We propose a classification method for cloud-persistent areas with high temporal dynamics of land-cover types. First, compositing techniques are employed to create dense time-series composite images from all available Landsat 8 images. Then, spectral–temporal features are extracted to train an ensemble of five supervised classifiers. The resulting composite images are clear with at least 99.78% cloud-free pixels and are 20.47% better than their original images on average. We classify seven land classes, including paddy rice, cropland, grass/shrub, trees, bare land, impervious area, and waterbody over Hanoi, Vietnam, in 2016. Using a time series of composites significantly improves the classification performance with 10.03% higher overall accuracy (OA) compared to single composite classifications. Additionally, using time series of composites and the ensemble technique, which combines the best of five experimented classifiers (eXtreme Gradient Boosting, logistic regression, Support Vector Machine (SVM) with Radial Basis Function (RBF) kernel – SVM–RBF and Linear kernel – SVM–Linear, multilayer perceptron), performed best with 84% OA and 0.79 kappa coefficient.     

样本特征对参数/非参数分类器分类精度的影响分析

为验证理论训练数量(10~30 p)对参数分类器(如最大似然分类)、非参数分类器(如支撑向量机)的适用性以及样本特征(光谱统计、空间分布特征)对分类器分类精度的影响,选择不同规模的训练样本进行最大似然分类和支撑向量机分类,分析分类精度与样本之间的关系。实验结果表明:随着样本量的增加,最大似然、支撑向量机分类精度均随样本量增多而提高并趋于稳定,最大似然分类精度的增长速度要快于支撑向量机。MLC受样本量的影响较大,在小样本的时候(5个),分类精度不稳定,超过30个样本的时候,分类精度稳定下来;对于SVM分类器,在小样本的时候(5个),分类精度较高且稳定,因此SVM分类适合于小样本分类,不受限于理论样本量的影响。当样本量超过最小理论样本量值(30个)的时候,最大似然分类精度要优于支撑向量机,主要是由于当样本量增加后,最大似然更易于获得有效的信息量样本,而对于支撑向量机边缘信息样本的增加数量不大。研究结果为进一步优化样本进行分类打下前期的实验基础。