A hybrid discriminative/generative approach to protein fold recognition

There are two standard approaches to the classification task: generative, which use training data to estimate a probability model for each class, and discriminative, which try to construct flexible decision boundaries between the classes. An ideal classifier should combine these two approaches. In this paper a classifier combining the well-known support vector machine (SVM) classifier with regularized discriminant analysis (RDA) classifier is presented. The hybrid classifier is used for protein structure prediction which is one of the most important goals pursued by bioinformatics. The obtained results are promising, the hybrid classifier achieves better result than the SVM or RDA classifiers alone. The proposed method achieves higher recognition ratio than other methods described in the literature.     

Benchmarking Least Squares Support Vector Machine Classifiers

In Support Vector Machines (SVMs), the solution of the classification problem is characterized by a (convex) quadratic programming (QP) problem. In a modified version of SVMs, called Least Squares SVM classifiers (LS-SVMs), a least squares cost function is proposed so as to obtain a linear set of equations in the dual space. While the SVM classifier has a large margin interpretation, the LS-SVM formulation is related in this paper to a ridge regression approach for classification with binary targets and to Fisher's linear discriminant analysis in the feature space. Multiclass categorization problems are represented by a set of binary classifiers using different output coding schemes. While regularization is used to control the effective number of parameters of the LS-SVM classifier, the sparseness property of SVMs is lost due to the choice of the 2-norm. Sparseness can be imposed in a second stage by gradually pruning the support value spectrum and optimizing the hyperparameters during the sparse approximation procedure. In this paper, twenty public domain benchmark datasets are used to evaluate the test set performance of LS-SVM classifiers with linear, polynomial and radial basis function (RBF) kernels. Both the SVM and LS-SVM classifier with RBF kernel in combination with standard cross-validation procedures for hyperparameter selection achieve comparable test set performances. These SVM and LS-SVM performances are consistently very good when compared to a variety of methods described in the literature including decision tree based algorithms, statistical algorithms and instance based learning methods. We show on ten UCI datasets that the LS-SVM sparse approximation procedure can be successfully applied.     

Confidence-based classifier design

In this paper, a new classifier design methodology, confidence-based classifier design, is proposed to design classifiers with controlled confidence. This methodology is under the guidance of two optimal classification theories, a new classification theory for designing optimal classifiers with controlled error rates and the C.K. Chow's optimal classification theory for designing optimal classifiers with controlled conditional error. The new methodology also takes advantage of the current well-developed classifier's probability preserving and ordering properties. It calibrates the output scores of current classifiers to the conditional error or error rates. Thus, it can either classify input samples or reject them according to the output scores of classifiers. It can achieve some reasonable performance even though it is not an optimal solution. An example is presented to implement the new methodology using support vector machines (SVMs). The empirical cumulative density function method is used to estimate error rates from the output scores of a trained SVM. Furthermore, a new dynamic bin width allocation method is proposed to estimate sample conditional error and this method adapts to the underlying probabilities. The experimental results clearly demonstrate the efficacy of the suggested classifier design methodology.     

基于最大熵估计的支持向量机概率建模

提出一种基于最大熵估计的支持向量机概率建模方法．针对传统的支持向量机方法不能提供后验概率的输出问题,从信息熵的角度采用最大熵估计方法,直接对支持向量机输出进行后验概率建模．实验结果表明,与同类算法相比,所提出的基于最大熵估计的概率建模方法具有优良的性能．     

一种多类支持向量机概率建模新方法

在支持向量机多类分类问题输出概率建模中,提出了一种直接求解后验概率的概率建模新方法。在对多个两类支持向量机分类器的输出概率进行组合时,该方法充分考虑了各个两类支持向量机分类器的差异,并以后验概率作为各个两类支持向量机分类器的权系数。仿真图像的实验结果表明,该文提出的直接求解后验概率方法与投票法及Pairwise Coupling方法相比,不仅具有较好的分类性能,而且得到的后验概率具有较好的概率分布形态。     

Classifying the Geometric Dilution of Precision of GPS satellites utilizing Bayesian decision theory

The errors resulting from satellite configuration geometry can be determined by Geometric Dilution of Precision (GDOP). Considering optimal satellite subset selection, lower GDOP value usually causes better accuracy in GPS positioning. However, GDOP computation based on complicated transformation and inversion of measurement matrices is a time consuming procedure. This paper deals with classification of GPS GDOP utilizing Parzen estimation based Bayesian decision theory. The conditional probability of each class is estimated by Parzen algorithm. Then based on Bayesian decision theory, the class with maximum posterior probability is selected. The experiments on measured dataset demonstrate that the proposed algorithm lead, in mean classification improvement, to 4.08% in comparison with Support Vector Machine (SVM) and 9.83% in comparison with K-Nearest Neighbour (KNN) classifier. Extra work on feature extraction has been performed based on Principle Component Analysis (PCA). The results demonstrate that the feature extraction approach has best performance respect to all classifiers.     

Imbalanced data classification based on scaling kernel-based support vector machine

In many classification problems, the class distribution is imbalanced. Learning from the imbalance data is a remarkable challenge in the knowledge discovery and data mining field. In this paper, we propose a scaling kernel-based support vector machine (SVM) approach to deal with the multi-class imbalanced data classification problem. We first use standard SVM algorithm to gain an approximate hyperplane. Then, we present a scaling kernel function and calculate its parameters using the chi-square test and weighting factors. Experimental results on KEEL data sets show the proposed algorithm can resolve the classifier performance degradation problem due to data skewed distribution and has a good generalization.     

Class-specific multiple classifiers scheme to recognize emotions from speech signals

Automatic emotion recognition from speech signals is one of the important research areas, which adds value to machine intelligence. Pitch, duration, energy and Mel-frequency cepstral coefficients (MFCC) are the widely used features in the field of speech emotion recognition. A single classifier or a combination of classifiers is used to recognize emotions from the input features. The present work investigates the performance of the features of Autoregressive (AR) parameters, which include gain and reflection coefficients, in addition to the traditional linear prediction coefficients (LPC), to recognize emotions from speech signals. The classification performance of the features of AR parameters is studied using discriminant, k-nearest neighbor (KNN), Gaussian mixture model (GMM), back propagation artificial neural network (ANN) and support vector machine (SVM) classifiers and we find that the features of reflection coefficients recognize emotions better than the LPC. To improve the emotion recognition accuracy, we propose a class-specific multiple classifiers scheme, which is designed by multiple parallel classifiers, each of which is optimized to a class. Each classifier for an emotional class is built by a feature identified from a pool of features and a classifier identified from a pool of classifiers that optimize the recognition of the particular emotion. The outputs of the classifiers are combined by a decision level fusion technique. The experimental results show that the proposed scheme improves the emotion recognition accuracy. Further improvement in recognition accuracy is obtained when the scheme is built by including MFCC features in the pool of features.     

SVM-based hierarchical architectures for handwritten Bangla character recognition

We propose support vector machine (SVM) based hierarchical classification schemes for recognition of handwritten Bangla characters. A comparative study is made among multilayer perceptron, radial basis function network and SVM classifier for this 45 class recognition problem. SVM classifier is found to outperform the other classifiers. A fusion scheme using the three classifiers is proposed which is marginally better than SVM classifier. It is observed that there are groups of characters having similar shapes. These groups are determined in two different ways on the basis of the confusion matrix obtained from SVM classifier. In the former, the groups are disjoint while they are overlapped in the latter. Another grouping scheme is proposed based on the confusion matrix obtained from neural gas algorithm. Groups are disjoint here. Three different two-stage hierarchical learning architectures (HLAs) are proposed using the three grouping schemes. An unknown character image is classified into a group in the first stage. The second stage recognizes the class within this group. Performances of the HLA schemes are found to be better than single stage classification schemes. The HLA scheme with overlapped groups outperforms the other two HLA schemes.     

基于D-vine Copula理论的贝叶斯分类器设计

高斯判别分析、朴素贝叶斯等传统贝叶斯分类方法在构建变量的联合概率分布时,往往会对变量间的相关性进行简化处理,从而使得贝叶斯决策理论中类条件概率密度的估计与实际数据之间存在一定的偏差.对此,结合Copula函数研究特征变量之间的相关性优化问题,设计基于D-vine Copula理论的贝叶斯分类器,主要目的是为了提高类条件概率密度估计的准确性.将变量的联合概率分布分解为一系列二元Copula函数与边缘概率密度函数的乘积,采用核函数方法对边缘概率密度进行估计 ,通过极大似然估计对二元Copula函数的参数分别进行优化,进而得到类条件概率密度函数的形式.将基于D-vine Copula理论的贝叶斯分类器应用到生物电信号的分类问题上,并对分类效果进行分析和验证.结果表明,所提出的方法在各项分类指标上均具备良好的性能.     

Commitment and typicality measures for the Self-Organizing Map

Soft classification using Kohonen's Self-Organizing Map (SOM) has not been explored as thoroughly as the Multi-Layer-Perceptron (MLP) neural network. In this paper, we propose two non-parametric algorithms for the SOM to provide soft classification outputs. These algorithms, which are labelling-frequency-based, are called SOM Commitment (SOM-C) and SOM Typicality (SOM-T), expressing in the first case the degree of commitment the classifier has for each class for a specific pixel and in the second case, how typical that pixel's reflectances are of those upon which the classifier was trained. To evaluate the two proposed algorithms, soft classifications of a Satellite Pour l'Observation de la Terre (SPOT) High Resolution Visible (HRV) image and an Airborne Visible Infrared Imaging Spectrometer (AVIRIS) image were undertaken. Both traditional soft classifiers, i.e. Bayesian posterior probability and Mahalanobis typicality classifier, and the most frequently used non-parametric neural network model, i.e. MLP, were used as a comparison. Principal-components analysis (PCA) was used to explore the relationship between these measures. Results indicate that great similarities exist between the SOM-C, MLP and the Bayesian posterior probability classifiers, while the SOM-T corresponds closely with Mahalanobis typicality probabilities. However, as implemented, they have the advantage of being non-parametric. The proposed measures significantly outperformed Bayesian and Mahalanobis classifiers when using the hyperspectral AVIRIS image.     

Z‐BAG: A CLASSIFICATION ENSEMBLE SYSTEM WITH POSTERIOR PROBABILISTIC OUTPUTS

Ensemble systems improve the generalization of single classifiers by aggregating the prediction of a set of base classifiers. Assessing classification reliability (posterior probability) is crucial in a number of applications, such as biomedical and diagnosis applications, where the cost of a misclassified input vector can be unacceptable high. Available methods are limited to either calibrate the posterior probability on an aggregated decision value or obtain a posterior probability for each base classifier and aggregate the result. We propose a method that takes advantage of the distribution of the decision values from the base classifiers to summarize a statistic which is subsequently used to generate the posterior probability. Three approaches are considered to fit the probabilistic output to the statistic: the standard Gaussian CDF, isotonic regression, and linear logistic. Even though this study focuses on a bagged support vector machine ensemble (Z ‐bag), our approach is not limited by the aggregation method selected, the choice of base classifiers, nor the statistic used. Performance is assessed on one artificial and 12 real‐world data sets from the UCI Machine Learning Repository. Our approach achieves comparable or better generalization on accuracy and posterior estimation to existing ensemble calibration methods although lowering computational cost.     

DEA-Based Piecewise Linear Discriminant Analysis

Nonlinear classification models have better classification performance than the linear classifiers. However, for many nonlinear classification problems, piecewise-linear discriminant functions can approximate nonlinear discriminant functions. In this study, we combine the algorithm of data envelopment analysis (DEA) with classification information, and propose a novel DEA-based classifier to construct a piecewise-linear discriminant function, in this classifier, the nonnegative conditions of DEA model are loosed and class information is added; Finally, experiments are performed using a UCI data set to demonstrate the accuracy and efficiency of the proposed model.     

RVM‐based multi‐class classification of remotely sensed data

The relevance vector machine (RVM), a Bayesian extension of the support vector machine (SVM), has considerable potential for the analysis of remotely sensed data. Here, the RVM is introduced and used to derive a multi‐class classification of land cover with an accuracy of 91.25%, a level comparable to that achieved by a suite of popular image classifiers including the SVM. Critically, however, the output of the RVM includes an estimate of the posterior probability of class membership. This output may be used to illustrate the uncertainty of the class allocations on a per‐case basis and help to identify possible routes to further enhance classification accuracy.     

Land-cover classification from multiple classifiers using decision fusion based on the probabilistic graphical model

A novel method of using different classification algorithms in an integrated manner by adaptively weighted decision level fusion was proposed. The proposed fusion scheme involves two steps. First, we processed the data using each classifier separately and provided probability estimations for each pixel of the considered classes. Then, the results are aggregated on the basis of the decision rule of probabilistic graphical model according to the capabilities of classifiers and ancillary information. The method was tested and validated through the Landsat 8 operational land imager data using two different classifiers, namely, maximum likelihood classifier and support vector machine. The proposed method provided higher accuracy improvement than the separate use of different classifiers and that complex landscapes, such as mountainous regions, have higher accuracy improvement than the relatively homogenous ones. Moreover, the method can handle more than two types of classifiers and effectively introduce additional ancillary information for adaptive weight selection. These findings can help promote our proposed method as an emerging approach for land-cover classification through remote sensing technology.     

基于后验概率的SVM决策树多类分类算法

后验概率支持向量机方法对孤立点和噪声具有鲁棒性,并且可以减少支持向量的数量,从而降低计算复杂度。因此,针对最近提出的快速分类算法c-BTS,引入样本的后验概率,提出了一种基于后验概率的SVM决策树算法P2BTS。实验结果证明,基于后验概率的支持向量机决策树P2BTS比c-BTS的分类精度更高,且所需的二类分类器个数减少,在一定程度上降低了P2BTS分类决策的时间和比较的次数,提高了分类效率。     

基于后验概率SVM决策树多类的分类算法

后验概率支持向量机方法对孤立点和噪声具有鲁棒性,并且可以减少支持向量的数量,从而降低计算复杂度。因此,针对最近提出的快速分类算法c-BTS,引入样本的后验概率,提出了一种基于后验概率的SVM决策树算法P2BTS。实验结果证明,基于后验概率的支持向量机决策树P2BTS比c-BTS的分类精度更高,且所需的二类分类器个数减少,在一定程度上降低了P2BTS分类决策的时间和比较的次数,提高了分类效率。     

Gait classification in children with cerebral palsy by Bayesian approach

Cerebral palsy (CP) is a non-progressive neuro-developmental condition that occurs in early childhood and is associated with a motor impairment, usually affecting mobility and posture. Automatic accurate identification of CP gait has many potential applications, for example, assistance in diagnosis, clinical decision-making and communication among the clinical professionals. In previous studies, support vector machine (SVM) and neural networks have been applied to classify CP gait patterns. However, one of the disadvantages of SVM and many neural network models is that given a gait sample, it only predicts a gait pattern class label without providing any estimate of the underlying probability, which is particularly important in computer aided diagnostics applications. The objective of this study is to first investigate different pattern classification paradigms in the automatic gait analysis and address the significance of Bayesian classifier model, and then give a comprehensive performances comparison. Bayesian classification is based on Bayes’ decision theory, which compute the probability of a given data point belonging to a class. Then among all classes, we choose the one that has the largest probability, and classify the data point as being of that class. Using a publicly available CP gait data set (68 normal healthy and 88 with spastic diplegia form of CP), different features including the two basic temporal-spatial gait parameters (stride length and cadence) have been experimented. Various hold-out and cross-validation testing show that the Bayesian model offers excellent classification performances compared with some popular classifiers such as random forest and multiple layer perceptron. With many advantages considered, Bayesian classifier model is very significant in establishing a clinical decision system for gait analysis.     

Improving the SVM gender classification accuracy using clustering and incremental learning

Gender recognition has been playing a very important role in various applications such as human–computer interaction, surveillance, and security. Nonlinear support vector machines (SVMs) were investigated for the identification of gender using the Face Recognition Technology (FERET) image face database. It was shown that SVM classifiers outperform the traditional pattern classifiers (linear, quadratic, Fisher linear discriminant, and nearest neighbour). In this context, this paper aims to improve the SVM classification accuracy in the gender classification system and propose new models for a better performance. We have evaluated different SVM learning algorithms; the SVM‐radial basis function with a 5% outlier fraction outperformed other SVM classifiers. We have examined the effectiveness of different feature selection methods. AdaBoost performs better than the other feature selection methods in selecting the most discriminating features. We have proposed two classification methods that focus on training subsets of images among the training images. Method 1 combines the outcome of different classifiers based on different image subsets, whereas method 2 is based on clustering the training data and building a classifier for each cluster. Experimental results showed that both methods have increased the classification accuracy.     

一种基于改进的支持向量机多分类器图像标注方法

针对多标签图像标注问题,提出一种改进的支持向量机多分类器图像标注方法。首先引入直方图交叉距离作为核函数,然后把传统支持向量机的输出值变换为样本到超平面的距离。基于这两点改进,采用一种特征选择方法,从众多的图像特征中,选择那些相互之间冗余度较小的视觉特征,分别建立分类器,最终形成以距离大小为判别依据的支持向量机多分类器模型。此外,在建立分类器时,考虑到训练图像中不同标签类样本分布的不均匀,引入了一个关于图像类标签的概率分布值做为分类器的权重系数。实验采用ImageCLEF提供的图像标注数据集,在其上的实验验证了所采用的特征选择算法和多分类模型的有效性,其标注精度要优于其他传统分类模型,并且,实验结果与最新的方法相比也具有一定的竞争力。