Support vector machines for interval discriminant analysis

The use of data represented by intervals can be caused by imprecision in the input information, incompleteness in patterns, discretization procedures, prior knowledge insertion or speed-up learning. All the existing support vector machine (SVM) approaches working on interval data use local kernels based on a certain distance between intervals, either by combining the interval distance with a kernel or by explicitly defining an interval kernel. This article introduces a new procedure for the linearly separable case, derived from convex optimization theory, inserting information directly into the standard SVM in the form of intervals, without taking any particular distance into consideration.     

Density-induced margin support vector machines

This paper proposes a new classifier called density-induced margin support vector machines (DMSVMs). DMSVMs belong to a family of SVM-like classifiers. Thus, DMSVMs inherit good properties from support vector machines (SVMs), e.g., unique and global solution, and sparse representation for the decision function. For a given data set, DMSVMs require to extract relative density degrees for all training data points. These density degrees can be taken as relative margins of corresponding training data points. Moreover, we propose a method for estimating relative density degrees by using the K nearest neighbor method. We also show the upper bound on the leave-out-one error of DMSVMs for a binary classification problem and prove it. Promising results are obtained on toy as well as real-world data sets.     

Face recognition using discriminant locality preserving projections based on maximum margin criterion

In this paper, we propose a new discriminant locality preserving projections based on maximum margin criterion (DLPP/MMC). DLPP/MMC seeks to maximize the difference, rather than the ratio, between the locality preserving between-class scatter and locality preserving within-class scatter. DLPP/MMC is theoretically elegant and can derive its discriminant vectors from both the range of the locality preserving between-class scatter and the range space of locality preserving within-class scatter. DLPP/MMC can also derive its discriminant vectors from the null space of locality preserving within-class scatter when the parameter of DLPP/MMC approaches +∞. Experiments on the ORL, Yale, FERET, and PIE face databases show the effectiveness of the proposed DLPP/MMC.     

Face recognition using fuzzy maximum scatter discriminant analysis

As we know, classical Fisher discriminant analysis usually suffers from the small sample size problem due to the singularity problem of the within-class scatter matrix. In this paper, a novel fuzzy linear classifier, called fuzzy maximum scatter difference (FMSD) discriminant criterion, is proposed to extract features from samples, especially deals with outlier samples. FMSD takes the scatter difference between between-class and within-class as discriminant criterion, so it will not suffer from the small sample size problem. The conventional scatter difference discriminant criterion (SDDC) assumes the same level of relevance of each sample to the corresponding class. In this paper, the fuzzy set theory is introduced to the conventional SDDC algorithm, where the fuzzy k-nearest neighbor is adopted to achieve the distribution information of original samples. The distribution is utilized to redefine the scatter matrices that are different from the conventional SDDC and effective to extract discriminative features from outlier samples. Experiments conducted on FERET and ORL face databases demonstrate the effectiveness of the proposed method.     

基于Fisher鉴别分析的支持向量机训练样本缩减策略

提出一种用于支持向量机训练样本集的缩减策略。该策略运用Fisher鉴别分析方法快速地提取潜在的支持向量,并构成用于SVM的新的训练样本集。仿真实验表明,该算法能在保证不降低分类精度的前提下,对较大规模的样本进行有效的缩减,提高运算效率。     

Patent value analysis using support vector machines

Receiving patents or licenses is an inevitable act of research in order to protect new ideas leading innovation. Request for patents has increased exponentially in order to legalize the intellectual property. Measuring economical value of each patent has been widely studied in the literature. Majority of the research in this field is focused on the patent driver prospect handled for the patent offices. There are a variety of criteria affecting decisions on each patent right; and predicting the possibility of grant may help the researchers to take some precautions. Objective of this study is to propose a robust model to determine if the appeal has a chance of approval. A case study is run on the patents that are accepted and rejected in home appliance industry to construct an intelligent classification model. The support vector machine, Back-Propagation Network and Bayes classification methods are compared on the proposed model. The proposed model in this study will help the decision makers to predict whether the patent appeal will be accepted. The study is unique with the approach that helps the candidate patent owners.     

On-line voltage security assessment of power systems using core vector machines

This paper presents a core vector machine (CVM)-based algorithm for on-line voltage security assessment of power systems. To classify the system security status, a CVM has been trained for each contingency. The proposed CVM-based security assessment has very small training time and space in comparison with support vector machines (SVM) and artificial neural networks (ANNs)-based algorithms. The proposed algorithm produces less support vectors (SV) and therefore is faster than existing algorithms. In this paper, a new decision tree (DT)-based feature selection technique has been presented, too. The proposed CVM algorithm has been applied to New England 39-bus power system. The simulation results show the effectiveness and the stability of the proposed method for on-line voltage security assessment procedure of large-scale power system.     

平均邻近间隔支撑向量机

近年来,支撑向量机(SVMs)作为一种学习方法已经在机器学习和模式识别的研究领域中得到了成功的运用,然而传统SVMs缺少考虑数据的局部信息.文中将邻近间隔的基本思想引入到SVMs中,提出了平均邻近间隔支撑向量机(ANMSVMs).ANMSVMs继承了传统SVMs的优点,同时又充分利用了数据的局部信息,从而实现了泛化能力的进一步提高.人造数据和真实数据集的实验结果验证了该方法的有效性,并且相对于传统SVMs体现出了更强的泛化能力.     

结合独立成分分析和核向量机进行人脸识别

在人脸识别过程中,首先利用独立成分分析得到独立的人脸基影像,所提取的特征就是人脸图像在基影像上的投影系数,通过选择合适的特征个数可以达到较高的识别准确率。然后采用支持向量机和核向量机分别对待识别图像在基影像上的投影系数进行分类判决,结果显示二者都能达到较高的识别准确率,但随着特征个数的增加,核向量机的准确率更高,训练时间更短,支持向量更少。实验表明方法可行有效的。     

基于独立成分分析和核向量机的虹膜识别方法

针对虹膜识别过程中的特征提取及识别问题,提出了用独立成分分析提取虹膜特征,用核向量机进行识别的方法.从采集到的人眼图像中定位虹膜,并对其进行归一化处理和图像增强处理.用独立成分分析提取统计独立的特征,通过选择合适的特征个数可以达到较高的识别准确率.在得到虹膜特征编码后,用核向量机进行分类判决,核向量机是一种适合大规模数据集的快速支持向量机训练算法,并将结果与支持向量机的分类结果进行了对比.实验结果表明了该方法的可行性和有效性.     

Deep kernel learning in core vector machines

In machine learning literature, deep learning methods have been moving toward greater heights by giving due importance in both data representation and classification methods. The recently developed multilayered arc-cosine kernel leverages the possibilities of extending deep learning features into the kernel machines. Even though this kernel has been widely used in conjunction with support vector machines (SVM) on small-size datasets, it does not seem to be a feasible solution for the modern real-world applications that involve very large size datasets. There are lot of avenues where the scalability aspects of deep kernel machines in handling large dataset need to be evaluated. In machine learning literature, core vector machine (CVM) is being used as a scaling up mechanism for traditional SVMs. In CVM, the quadratic programming problem involved in SVM is reformulated as an equivalent minimum enclosing ball problem and then solved by using a subset of training sample (Core Set) obtained by a faster \((1+\epsilon )\) approximation algorithm. This paper explores the possibilities of using principles of core vector machines as a scaling up mechanism for deep support vector machine with arc-cosine kernel. Experiments on different datasets show that the proposed system gives high classification accuracy with reasonable training time compared to traditional core vector machines, deep support vector machines with arc-cosine kernel and deep convolutional neural network.     

基于最大间隔准则的鲁棒多流形判别局部图嵌入算法

针对现有的多流形人脸识别算法大多直接使用带有噪声的原始数据进行处理，而带有噪声的数据往往会对算法的准确率产生负面影响的问题，提出了一种基于最大间距准则的鲁棒多流形判别局部图嵌入算法（RMMDLGE/MMC）。首先，通过引入一个降噪投影对原始数据进行迭代降噪处理，提取出更加纯净的数据；其次，对数据图像进行分块，建立多流形模型；再次，结合最大间隔准则的思想，寻求最优的投影矩阵使得不同流形上的样本距离尽可能大，同时相同流形上的样本距离尽可能小；最后，计算待识样本流形到训练样本流形的距离进行分类识别。实验结果表明，与表现较好的最大间距准则框架下的多流形局部图嵌入算法（MLGE/MMC）相比，所提算法在添加噪声的ORL、Yale和FERET库上的分类识别率分别提高了1.04、1.28和2.13个百分点，分类效果明显提高。     

Primal explicit max margin feature selection for nonlinear support vector machines

Embedding feature selection in nonlinear support vector machines (SVMs) leads to a challenging non-convex minimization problem, which can be prone to suboptimal solutions. This paper develops an effective algorithm to directly solve the embedded feature selection primal problem. We use a trust-region method, which is better suited for non-convex optimization compared to line-search methods, and guarantees convergence to a minimizer. We devise an alternating optimization approach to tackle the problem efficiently, breaking it down into a convex subproblem, corresponding to standard SVM optimization, and a non-convex subproblem for feature selection. Importantly, we show that a straightforward alternating optimization approach can be susceptible to saddle point solutions. We propose a novel technique, which shares an explicit margin variable to overcome saddle point convergence and improve solution quality. Experiment results show our method outperforms the state-of-the-art embedded SVM feature selection method, as well as other leading filter and wrapper approaches.     

Radius margin bounds for support vector machines with the RBF kernel

An important approach for efficient support vector machine (SVM) model selection is to use differentiable bounds of the leave-one-out (loo) error. Past efforts focused on finding tight bounds of loo (e.g., radius margin bounds, span bounds). However, their practical viability is still not very satisfactory. Duan, Keerthi, and Poo (2003) showed that radius margin bound gives good prediction for L2-SVM, one of the cases we look at. In this letter, through analyses about why this bound performs well for L2-SVM, we show that finding a bound whose minima are in a region with small loo values may be more important than its tightness. Based on this principle, we propose modified radius margin bounds for L1-SVM (the other case) where the original bound is applicable only to the hard-margin case. Our modification for L1-SVM achieves comparable performance to L2-SVM. To study whether L1- or L2-SVM should be used, we analyze other properties, such as their differentiability, number of support vectors, and number of free support vectors. In this aspect, L1-SVM possesses the advantage of having fewer support vectors. Their implementations are also different, so we discuss related issues in detail.     

Face recognition using Gabor-based direct linear discriminant analysis and support vector machine

This paper presents a novel and uniform framework for face recognition. This framework is based on a combination of Gabor wavelets, direct linear discriminant analysis (DLDA) and support vector machine (SVM). First, feature vectors are extracted from raw face images using Gabor wavelets. These Gabor-based features are robust against local distortions caused by the variance of illumination, expression and pose. Next, the extracted feature vectors are projected to a low-dimensional subspace using DLDA technique. The Gabor-based DLDA feature vectors are then applied to SVM classifier. A new kernel function for SVM called hyperhemispherically normalized polynomial (HNP) is also proposed in this paper and its validity on the improvement of classification accuracy is theoretically proved and experimentally tested for face recognition. The proposed algorithm was evaluated using the FERET database. Experimental results show that the proposed face recognition system outperforms other related approaches in terms of recognition rate.     

Robust large margin discriminant tangent analysis for face recognition

Fisher’s Linear Discriminant Analysis (LDA) has been recognized as a powerful technique for face recognition. However, it could be stranded in the non-Gaussian case. Nonparametric discriminant analysis (NDA) is a typical algorithm that extends LDA from Gaussian case to non-Gaussian case. However, NDA suffers from outliers and unbalance problems, which cause a biased estimation of the extra-class scatter information. To address these two problems, we propose a robust large margin discriminant tangent analysis method. A tangent subspace-based algorithm is first proposed to learn a subspace from a set of intra-class and extra-class samples which are distributed in a balanced way on the local manifold patch near each sample point, so that samples from the same class are clustered as close as possible and samples from different classes will be separated far away from the tangent center. Then each subspace is aligned to a global coordinate by tangent alignment. Finally, an outlier detection technique is further proposed to learn a more accurate decision boundary. Extensive experiments on challenging face recognition data set demonstrate the effectiveness and efficiency of the proposed method for face recognition. Compared to other nonparametric methods, the proposed one is more robust to outliers.     

基于边界鉴别分析的递归维数约简算法

提出一种基于边界鉴别分析的递归维数约简算法.该算法把已求取边界鉴别向量正交于待求超平面法向量作为支持向量机(SVM)优化问题新的约束条件;然后对改进SVM进行递归求解,得到正交边界鉴别向量基;最后将数据样本在正交边界鉴别向量上投影实现维数约简.该算法不仅克服了现有维数约简算法难以支持小样本数据集、受数据样本分布影响等问题,而且抽取的特征向量具有更优的分类性能.仿真实验说明了算法的有效性.     

Human motion recognition using support vector machines

We propose a motion recognition strategy that represents each videoclip by a set of filtered images, each of which corresponds to a frame. Using a filtered-image classifier based on support vector machines, we classify a videoclip by applying majority voting over the predicted labels of its filtered images and, for online classification, we identify the most likely type of action at any moment by applying majority voting over the predicted labels of the filtered images within a sliding window. We also define a classification confidence and the associated threshold in both cases, which enable us to identify the existence of an unknown type of motion and, together with the proposed recognition strategy, make it possible to build a real-time motion recognition system that cannot only make classifications in real-time, but also learn new types of motions and recognize them in the future. The proposed strategy is demonstrated on real datasets.     

Gesture phase segmentation using support vector machines

An interaction between humans or between a human and a machine will be more effective if it is supported by gestures. In different levels of complexity, the communication system used in human interaction includes the use of gesture. In natural conversation, for instance, speakers use gestures for both to enhance the expressiveness of their speech and to support their own linguistic reasoning. The audience absorbs the content being transmitted also based on the speakers’ gesticulation. Thus, an analysis of gestures should add value to the purpose of the interaction. One of the concerns in the analysis of gestures is the problem arising from the segmentation of phases of a gesture (rest position, preparation, stroke, hold and retraction), which, from the standpoint of Gesture Theory, may reveal information on prosody and semantics of what is being said in a discourse. Finding an automation solution to this problem involves enabling the development of theoretical and application areas that are based on the analysis of human behavior and on the interpretation and generation of natural language. In this study, the problem of gesture phase segmentation is modeled as a problem of classification, and then support vector machine is employed to design a model able to learn the patterns of gesture that are inherent to each phase. This work presents two main highlights. The first is to address the limitations of the segmentation approach through the study of its performance in different scenarios that represent the complexity of analyzing patterns of human behavior. In this study, we reached an F-score around 0.9 for rest position and around 0.8 for stroke and preparation as segmentation results in the best cases. Moreover, it was possible to investigate how classification models are influenced by human behavior. The second highlight refers to the conduction of an analysis by considering the standpoint of specialists concerned with gesture phase segmentation in the area of Linguistics and Psycholinguistics, through which we obtained impressive results. Thus, in regard to the suitability of our approach, it is a feasible means of supporting the development of the Gesture Theory as well as the Computational Linguistics and Human Machine Interaction fields.