Support vector machines for texture classification

This paper investigates the application of support vector machines (SVMs) in texture classification. Instead of relying on an external feature extractor, the SVM receives the gray-level values of the raw pixels, as SVMs can generalize well even in high-dimensional spaces. Furthermore, it is shown that SVMs can incorporate conventional texture feature extraction methods within their own architecture, while also providing solutions to problems inherent in these methods. One-against-others decomposition is adopted to apply binary SVMs to multitexture classification, plus a neural network is used as an arbitrator to make final classifications from several one-against-others SVM outputs. Experimental results demonstrate the effectiveness of SVMs in texture classification.     

支持向量机分类算法研究

支持向量机在处理两类分类问题时,当两类样本混杂严重时会降低分类精度。在NN-SVM分类算法的基础上,通过计算样本点与其最近邻点类别的异同以及该点与其k个同类近邻点在核空间的平均距离修剪混淆点,进而提出了一种改进的NN-SVM算法——KCNN-SVM。实验数据表明,KCNN-SVM算法与SVM以及NN-SVM相比,有着更高的分类精度和更快的训练、分类时间。     

Support vector machines of interval-based features for time series classification

In previous works, a time series classification system has been presented. It is based on boosting very simple classifiers, formed only by one literal. The used literals are based on temporal intervals.The obtained classifiers were simply a linear combination of literals, so it is natural to expect some improvements in the results if those literals were combined in more complex ways. In this work we explore the possibility of using the literals selected by the boosting algorithm as new features, and then using a SVM with these metafeatures. The experimental results show the validity of the proposed method.     

Content-based image classification with wavelet relevance vector machines

This paper introduces the use of relevance vector machines (RVMs) for content-based image classification and compares it with the conventional support vector machine (SVM) approach. Different wavelet kernels are included in the formulation of the RVM. We also propose a new wavelet-based feature extraction method that extracts lesser number of features as compared to other wavelet-based feature extraction methods. Experimental results confirm the superiority of RVM over SVM in terms of the trade-off between slightly reduced accuracy but substantially enhanced sparseness of the solution, and also the ease of free parameters tuning.     

结合超像元和子空间投影支持向量机的高光谱图像分类

目的 高光谱图像包含了丰富的空间、光谱和辐射信息,能够用于精细的地物分类,但是要达到较高的分类精度,需要解决高维数据与有限样本之间存在矛盾的问题,并且降低因噪声和混合像元引起的同物异谱的影响。为有效解决上述问题,提出结合超像元和子空间投影支持向量机的高光谱图像分类方法。方法 首先采用简单线性迭代聚类算法将高光谱图像分割成许多无重叠的同质性区域,将每一个区域作为一个超像元,以超像元作为图像分类的最小单元,利用子空间投影算法对超像元构成的图像进行降维处理,在低维特征空间中执行支持向量机分类。本文高光谱图像空谱综合分类模型,对几何特征空间下的超像元分割与光谱特征空间下的子空间投影支持向量机（SVMsub）,采用分割后进行特征融合的处理方式,将像元级别转换为面向对象的超像元级别,实现高光谱图像空谱综合分类。结果 在AVIRIS（airbone visible/infrared imaging spectrometer）获取的Indian Pines数据和Reflective ROSIS（optics system spectrographic imaging system）传感器获取的University of Pavia数据实验中,子空间投影算法比对应的非子空间投影算法的分类精度高,特别是在样本数较少的情况下,分类效果提升明显;利用马尔可夫随机场或超像元融合空间信息的算法比对应的没有融合空间信息的算法的分类精度高;在两组数据均使用少于1%的训练样本情况下,同时融合了超像元和子空间投影的支持向量机算法在两组实验中分类精度均为最高,整体分类精度高出其他相关算法4%左右。结论 利用超像元处理可以有效融合空间信息,降低同物异谱对分类结果的不利影响;采用子空间投影能够将高光谱数据变换到低维空间中,实现有限训练样本条件下的高精度分类;结合超像元和子空间投影支持向量机的算法能够得到较高的高光谱图像分类精度。     

Fuzzy-rough feature selection aided support vector machines for Mars image classification

This paper presents a novel application of advanced machine learning techniques for Mars terrain image classification. Fuzzy-rough feature selection (FRFS) is adapted and then employed in conjunction with Support Vector Machines (SVMs) to construct image classifiers. These techniques are integrated to address problems in space engineering where the images are of many classes, large-scale, and diverse representational properties. The use of the adapted FRFS allows the induction of low-dimensionality feature sets from feature patterns of a much higher dimensionality. To evaluate the proposed work, K-Nearest Neighbours (KNNs) and decision trees (DTREEs) based image classifiers as well as information gain rank (IGR) based feature selection are also investigated here, as possible alternatives to the underlying machine learning techniques adopted. The results of systematic comparative studies demonstrate that in general, feature selection improves the performance of classifiers that are intended for use in high dimensional domains. In particular, the proposed approach helps to increase the classification accuracy, while enhancing classification efficiency by requiring considerably less features. This is evident in that the resultant SVM-based classifiers which utilise FRFS-selected features generally outperform KNN and DTREE based classifiers and those which use IGR-returned features. The work is therefore shown to be of great potential for on-board or ground-based image classification in future Mars rover missions.     

Additive support vector machines for pattern classification.

Support vector machines (SVMs) are one of the most popular methodologies for the design of pattern classification systems with sound theoretical foundations and high generalizing performance. The SVM framework focuses on linear and nonlinear models that maximize the separating margin between objects belonging in different classes. This paper extends the SVM modeling context toward the development of additive models that combine the simplicity and transparency/interpretability of linear classifiers with the generalizing performance of nonlinear models. Experimental results are also presented on the performance of the new methodology over existing SVM techniques.     

Support vector machines with genetic fuzzy feature transformation for biomedical data classification

In this paper, we present a genetic fuzzy feature transformation method for support vector machines (SVMs) to do more accurate data classification. Given data are first transformed into a high feature space by a fuzzy system, and then SVMs are used to map data into a higher feature space and then construct the hyperplane to make a final decision. Genetic algorithms are used to optimize the fuzzy feature transformation so as to use the newly generated features to help SVMs do more accurate biomedical data classification under uncertainty. The experimental results show that the new genetic fuzzy SVMs have better generalization abilities than the traditional SVMs in terms of prediction accuracy.     

Support vector machines for face recognition

Support vector machines (SVMs) have been recently proposed as a new learning network for bipartite pattern recognition. In this paper, SVMs incorporated with a binary tree recognition strategy are proposed to tackle the multi-class face recognition problem. The binary tree extends naturally, the pairwise discrimination capability of the SVMs to the multi-class scenario. Two face databases are used to evaluate the proposed method. The performance of the SVMs based face recognition is compared with the standard eigenface approach, and also the more recently proposed algorithm called the nearest feature line (NFL).     

Support vector machines for dyadic data

We describe a new technique for the analysis of dyadic data, where two sets of objects (row and column objects) are characterized by a matrix of numerical values that describe their mutual relationships. The new technique, called potential support vector machine (P-SVM), is a large-margin method for the construction of classifiers and regression functions for the column objects. Contrary to standard support vector machine approaches, the P-SVM minimizes a scale-invariant capacity measure and requires a new set of constraints. As a result, the P-SVM method leads to a usually sparse expansion of the classification and regression functions in terms of the row rather than the column objects and can handle data and kernel matrices that are neither positive definite nor square. We then describe two complementary regularization schemes. The first scheme improves generalization performance for classification and regression tasks; the second scheme leads to the selection of a small, informative set of row support objects and can be applied to feature selection. Benchmarks for classification, regression, and feature selection tasks are performed with toy data as well as with several real-world data sets. The results show that the new method is at least competitive with but often performs better than the benchmarked standard methods for standard vectorial as well as true dyadic data sets. In addition, a theoretical justification is provided for the new approach.     

Support vector machines for spam categorization

We study the use of support vector machines (SVM) in classifying e-mail as spam or nonspam by comparing it to three other classification algorithms: Ripper, Rocchio, and boosting decision trees. These four algorithms were tested on two different data sets: one data set where the number of features were constrained to the 1000 best features and another data set where the dimensionality was over 7000. SVM performed best when using binary features. For both data sets, boosting trees and SVM had acceptable test performance in terms of accuracy and speed. However, SVM had significantly less training time.     

Support vector machines for olfactory signals recognition

Pattern recognition techniques have widely been used in the context of odor recognition. The recognition of mixtures and simple odors as separate clusters is an untractable problem with some of the classical supervised methods. Recently, a new paradigm has been introduced in which the detection problem can be seen as a learning from examples problem. In this paper, we investigate odor recognition in this new perspective and in particular by using a novel learning scheme known as support vector machines (SVM) which guarantees high generalization ability on the test set. We illustrate the basics of the theory of SVM and show its performance in comparison with radial basis network and the error backpropagation training method. The leave-one-out procedure has been used for all classifiers, in order to finding the near-optimal SVM parameter and both to reduce the generalization error and to avoid outliers.     

Support vector machines for urban growth modeling

This paper presents a novel method to model urban land use conversion using support vector machines (SVMs), a new generation of machine learning algorithms used in the classification and regression domains. This method derives the relationship between rural-urban land use change and various factors, such as population, distance to road and facilities, and surrounding land use. Our study showed that SVMs are an effective approach to estimating the land use conversion model, owing to their ability to model non-linear relationships, good generalization performance, and achievement of a global and unique optimum. The rural-urban land use conversions of New Castle County, Delaware between 1984–1992, 1992–1997, and 1997–2002 were used as a case study to demonstrate the applicability of SVMs to urban expansion modeling. The performance of SVMs was also compared with a commonly used binomial logistic regression (BLR) model, and the results, in terms of the overall modeling accuracy and McNamara’s test, consistently corroborated the better performance of SVMs.     

Support vector machines for spatiotemporal tornado prediction

The use of support vector machines (SVMs) for predicting the location and time of tornadoes is presented. In this paper, we extend the work by Lakshmanan et al. (Proceedings of 2005 IEEE international joint conference on neural networks (Montreal, Canada), 3, 2005a, 1642–1647) to use a set of 33 storm days and introduce some variations that improve the results. The goal is to estimate the probability of a tornado event at a particular spatial location within a given time window. We utilize a least-squares methodology to estimate shear, quality control of radar reflectivity, morphological image processing to estimate gradients, fuzzy logic to generate compact measures of tornado possibility and SVM classification to generate the final spatiotemporal probability field. On the independent test set, this method achieves a Heidke's skill score of 0.60 and a critical success index of 0.45.     

Support vector machines for interval discriminant analysis

The use of data represented by intervals can be caused by imprecision in the input information, incompleteness in patterns, discretization procedures, prior knowledge insertion or speed-up learning. All the existing support vector machine (SVM) approaches working on interval data use local kernels based on a certain distance between intervals, either by combining the interval distance with a kernel or by explicitly defining an interval kernel. This article introduces a new procedure for the linearly separable case, derived from convex optimization theory, inserting information directly into the standard SVM in the form of intervals, without taking any particular distance into consideration.     

Efficient sparse nonparallel support vector machines for classification

In this paper, we propose a novel nonparallel classifier, named sparse nonparallel support vector machine (SNSVM), for binary classification. Different with the existing nonparallel classifiers, such as the twin support vector machines (TWSVMs), SNSVM has several advantages: It constructs two convex quadratic programming problems for both linear and nonlinear cases, which can be solved efficiently by successive overrelaxation technique; it does not need to compute the inverse matrices any more before training; it has the similar sparseness with standard SVMs; it degenerates to the TWSVMs when the parameters are appropriately chosen. Therefore, SNSVM is certainly superior to them theoretically. Experimental results on lots of data sets show the effectiveness of our method in both sparseness and classification accuracy and, therefore, confirm the above conclusions further.     

Support vector machines for 3D object recognition

Support vector machines (SVMs) have been recently proposed as a new technique for pattern recognition. Intuitively, given a set of points which belong to either of two classes, a linear SVM finds the hyperplane leaving the largest possible fraction of points of the same class on the same side, while maximizing the distance of either class from the hyperplane. The hyperplane is determined by a subset of the points of the two classes, named support vectors, and has a number of interesting theoretical properties. In this paper, we use linear SVMs for 3D object recognition. We illustrate the potential of SVMs on a database of 7200 images of 100 different objects. The proposed system does not require feature extraction and performs recognition on images regarded as points of a space of high dimension without estimating pose. The excellent recognition rates achieved in all the performed experiments indicate that SVMs are well-suited for aspect-based recognition     

Support vector machines for speaker and language recognition

Support vector machines (SVMs) have proven to be a powerful technique for pattern classification. SVMs map inputs into a high-dimensional space and then separate classes with a hyperplane. A critical aspect of using SVMs successfully is the design of the inner product, the kernel, induced by the high dimensional mapping. We consider the application of SVMs to speaker and language recognition. A key part of our approach is the use of a kernel that compares sequences of feature vectors and produces a measure of similarity. Our sequence kernel is based upon generalized linear discriminants. We show that this strategy has several important properties. First, the kernel uses an explicit expansion into SVM feature space—this property makes it possible to collapse all support vectors into a single model vector and have low computational complexity. Second, the SVM builds upon a simpler mean-squared error classifier to produce a more accurate system. Finally, the system is competitive and complimentary to other approaches, such as Gaussian mixture models (GMMs). We give results for the 2003 NIST speaker and language evaluations of the system and also show fusion with the traditional GMM approach.     

Support vector machines and malware detection

In this research, we test three advanced malware scoring techniques that have shown promise in previous research, namely, Hidden Markov Models, Simple Substitution Distance, and Opcode Graph based detection. We then perform a careful robustness analysis by employing morphing strategies that cause each score to fail. We show that combining scores using a Support Vector Machine yields results that are significantly more robust than those obtained using any of the individual scores.