Minimum class variance support vector machines.

In this paper, a modified class of support vector machines (SVMs) inspired from the optimization of Fisher's discriminant ratio is presented, the so-called minimum class variance SVMs (MCVSVMs). The MCVSVMs optimization problem is solved in cases in which the training set contains less samples that the dimensionality of the training vectors using dimensionality reduction through principal component analysis (PCA). Afterward, the MCVSVMs are extended in order to find nonlinear decision surfaces by solving the optimization problem in arbitrary Hilbert spaces defined by Mercer's kernels. In that case, it is shown that, under kernel PCA, the nonlinear optimization problem is transformed into an equivalent linear MCVSVMs problem. The effectiveness of the proposed approach is demonstrated by comparing it with the standard SVMs and other classifiers, like kernel Fisher discriminant analysis in facial image characterization problems like gender determination, eyeglass, and neutral facial expression detection.     

Kernel-based adversarial attacks and defenses on support vector classification

While malicious samples are widely found in many application fields of machine learning, suitable countermeasures have been investigated in the field of adversarial machine learning. Due to the importance and popularity of Support Vector Machines (SVMs), we first describe the evasion attack against SVM classification and then propose a defense strategy in this paper. The evasion attack utilizes the classification surface of SVM to iteratively find the minimal perturbations that mislead the nonlinear classifier. Specially, we propose what is called a vulnerability function to measure the vulnerability of the SVM classifiers. Utilizing this vulnerability function, we put forward an effective defense strategy based on the kernel optimization of SVMs with Gaussian kernel against the evasion attack. Our defense method is verified to be very effective on the benchmark datasets, and the SVM classifier becomes more robust after using our kernel optimization scheme.     

Comparing support vector machines with Gaussian kernels to radialbasis function classifiers

The support vector (SV) machine is a novel type of learning machine, based on statistical learning theory, which contains polynomial classifiers, neural networks, and radial basis function (RBF) networks as special cases. In the RBF case, the SV algorithm automatically determines centers, weights, and threshold that minimize an upper bound on the expected test error. The present study is devoted to an experimental comparison of these machines with a classical approach, where the centers are determined by X-means clustering, and the weights are computed using error backpropagation. We consider three machines, namely, a classical RBF machine, an SV machine with Gaussian kernel, and a hybrid system with the centers determined by the SV method and the weights trained by error backpropagation. Our results show that on the United States postal service database of handwritten digits, the SV machine achieves the highest recognition accuracy, followed by the hybrid system. The SV approach is thus not only theoretically well-founded but also superior in a practical application     

Support vector machines for quality monitoring in a plastic injection molding process

Support vector machines (SVMs) are receiving increased attention in different application domains for which neural networks (NNs) have had a prominent role. However, in quality monitoring little attention has been given to this more recent development encompassing a technique with foundations in statistic learning theory. In this paper, we compare C-SVM and /spl nu/-SVM classifiers with radial basis function (RBF) NNs in data sets corresponding to product faults in an industrial environment concerning a plastics injection molding machine. The goal is to monitor in-process data as a means of indicating product quality and to be able to respond quickly to unexpected process disturbances. Our approach based on SVMs exploits the first part of this goal. Model selection which amounts to search in hyperparameter space is performed for study of suitable condition monitoring. In the multiclass problem formulation presented, classification accuracy is reported for both strategies. Experimental results obtained thus far indicate improved generalization with the large margin classifier as well as better performance enhancing the strength and efficacy of the chosen model for the practical case study.     

基于高阶相关的稀疏化和径向基核分类法

非平稳信号处理理论中高阶统计量方法被广泛应用于模式识别以提取稳健特征，但算法本身需要解决计算量大的问题，加上训练样本具有冗余性，因而限制了模式特征提取和分类速度。该文将计算高阶统计量转化为求相关系数，研究高阶相关在样本稀疏化以及基于径向基核的非线性分类方面的应用。首先采用最大匹配系数法确定相关阶数，然后对训练样本稀疏化，最后将高阶相关应用于SVM、KNR两种核非线性分类器进行分类识别，避免了高阶统计量的直接计算，减少了训练和分类时间。对手写数字和8种飞机的仿真数据进行实验，结果表明该方法具有较好的稀疏效果和识别效果。     

Use of Support Vector Regression in Stable Trajectory Generation for Walking Humanoid Robots

This paper concerns the use of support vector regression (SVR), which is based on the kernel method for learning from examples, in identification of walking robots. To handle complex dynamics in humanoid robot and realize stable walking, this paper develops and implements two types of reference natural motions for a humanoid, namely, walking trajectories on a flat floor and on an ascending slope. Next, SVR is applied to model stable walking motions by considering these actual motions. Three kinds of kernels, namely, linear, polynomial, and radial basis function (RBF), are considered, and the results from these kernels are compared and evaluated. The results show that the SVR approach works well, and SVR with the RBF kernel function provides the best performance. Plus, it can be effectively applied to model and control a practical biped walking robot.     

基于EMD-SVM的江水浊度预测方法研究

针对江水浊度序列宽频、非线性、非平稳的特点,将经验模态分解(EMD)和支持向量机(SVM)回归方法引入浊度预测领域,建立了基于EMD-SVM的浊度预测模型.通过EMD分解,将原始非平稳的浊度序列分解为若干固有模态分量(IMF),根据各IMF序列的特点,选择不同的参数对各IMF序列进行预测,最后合成原始序列的预测值.将该方法应用于实际浊度预测,并与径向基神经网络(RBF)预测及单独支持向量机回归预测结果进行比较,仿真结果表明该方法预测精度有明显提高.     

基于图像核函数的图像目标识别技术研究

核方法已经广泛应用于模式识别的各个领域,但是传统的核函数仅能接受一维矢量作为输入数据,对二维图像数据需要进行一定的预处理.本文从核函数构造的角度出发,以RBF核函数为结构原型,定义了一种输入量为图像数据的新型核函数.从形式上看,本核函数仅在输人数据的类型上有所不同,其他方面完全一致,经典的基于核方法的分类器算法都可以使用本核函数.对UCI数据实验结果表明,本核函数在多种图像识别应用中均有良好的效果.     

An enhanced feature selection filter for classification of microarray cancer data

The main aim of this study is to select the optimal set of genes from microarray cancer datasets that contribute to the prediction of specific cancer types. This study proposes the enhancement of the feature selection filter algorithm based on Joe's normalized mutual information and its use for gene selection. The proposed algorithm is implemented and evaluated on seven benchmark microarray cancer datasets, namely, central nervous system, leukemia (binary), leukemia (3 class), leukemia (4 class), lymphoma, mixed lineage leukemia, and small round blue cell tumor, using five well‐known classifiers, including the naive Bayes, radial basis function network, instance‐based classifier, decision‐based table, and decision tree. An average increase in the prediction accuracy of 5.1% is observed on all seven datasets averaged over all five classifiers. The average reduction in training time is 2.86 seconds. The performance of the proposed method is also compared with those of three other popular mutual information–based feature selection filters, namely, information gain, gain ratio, and symmetric uncertainty. The results are impressive when all five classifiers are used on all the datasets.     

隐相空间的DUPSO-RPSOVF语音预测模型研究

提出了一种基于二阶Volterra级数的语音信号非线性预测模型.为克服传统的最小均方（Least Mean Square,LMS）算法在模型核系数更新时的固有缺点,引入耗散均匀搜索粒子群优化算法（Dissipative Uniform Particle Swarm Optimization,DUPSO）求解核系数,并构建了DUPSO-SOVF预测模型;为避免传统方法中相空间的重构过程,构建了隐相空间DUPSO-SOVF预测模型,在求解模型核系数时动态地求解出最优嵌入维数和延迟时间;为降低模型复杂度,在误差允许范围内进行模型关键项的提取,从而减少了核系数个数,构建了少参数的DUPSO-RPSOVF（Reduced Parameter SOVF,RPSOVF）预测模型.将英语音素、单词和短语作为实验样本数据进行仿真,结果表明：隐相空间DUPSO-SOVF模型能够准确的计算出相空间重构参数,DUPSO-SOVF和DUPSO-RPSOVF两种预测模型对单帧和多帧语音信号均具有较高的预测精度,优于PSO-SOVF和LMS-SOVF预测模型,并且能够很好地反映语音序列变化的趋势和规律,可以满足语音序列预测的要求.     

Application of kernel principal component analysis for single-lead-ECG-derived respiration

Recent studies show that principal component analysis (PCA) of heartbeats is a well-performing method to derive a respiratory signal from ECGs. In this study, an improved ECG-derived respiration (EDR) algorithm based on kernel PCA (kPCA) is presented. KPCA can be seen as a generalization of PCA where nonlinearities in the data are taken into account by nonlinear mapping of the data, using a kernel function, into a higher dimensional space in which PCA is carried out. The comparison of several kernels suggests that a radial basis function (RBF) kernel performs the best when deriving EDR signals. Further improvement is carried out by tuning the parameter σ(2) that represents the variance of the RBF kernel. The performance of kPCA is assessed by comparing the EDR signals to a reference respiratory signal, using the correlation and the magnitude squared coherence coefficients. When comparing the coefficients of the tuned EDR signals using kPCA to EDR signals obtained using PCA and the algorithm based on the R peak amplitude, statistically significant differences are found in the correlation and coherence coefficients (both p<0.0001), showing that kPCA outperforms PCA and R peak amplitude in the extraction of a respiratory signal from single-lead ECGs.     

线性判别分析和支持向量机的音乐分类方法

提出了一种新的音乐分类方法,该方法使用线性判别分析(LDA)和支持向量机(SVMs)对音乐数据进行分类。在实现音乐分类中,先使用傅里叶变换等方法从每一段音乐中提取音频特征,包括Mel倒谱系数及基音频率等,并将它们按比例组成一个高维向量;再使用LDA对这些高维向量进行降维,使得各类音乐的类间离散度与类内离散度的比值最大;最后使用SVM等4种分类器对降维后的特征进行分类。实验证明LDA及SVM使得音乐分类的精确度有了较大的提高。     

Sentiment classification using the theory of ANNs

Sentiment classification has attracted increasing interest from natural language processing. The goal of sentiment classification is to automatically identify whether a given piece of text expresses positive or negative opinion on a topic of interest. This paper presents the standpoint that uses individual model (i-model) based on artificial neural networks (ANNs) to determine text sentiment classification. The individual model consists of sentimental features, feature weight and prior knowledge base. During the training process, i-model that makes right sentimental judgment will correct those are wrong, to make more accurate prediction of text sentiment polarity. Experimental results show that the accuracy of individual model is higher than that of support vector machines (SVMs) and hidden Markov model (HMM) classifiers on movie review corpus.     

基于偏振成像技术的油桃机械损伤检测

为了解决由于油桃表面颜色特征复杂所带来的早期机械损伤难以检测问题, 提出了一种基于偏振成像技术的早期损伤检测分类模型。采用分焦平面偏振成像方法一次性获取油桃在4个偏振方向下的偏振图像, 利用双线性插值和低照度增强(LIME)对偏振图像进行预处理, 以提高运行实时性并降低水果曲率变化的影响; 提取偏振图像中像素的颜色特征和灰度共生矩阵(GLCM)特征, 分别用于训练两个最小二乘支持向量机(LSSVM)分类模型; 通过理论分析和实验仿真, 最后利用两个分类模型的串联(color-LSSVM→GLCM-LSSVM model)实现了油桃机械损伤的早期检测。结果表明, 该分类器模型对油桃正常和损伤区域的检测精确率达到95.68%, 召回率达到93.29%。分焦平面偏振成像技术在深色系水果的早期损伤无损检测领域具有良好的应用前景。     

Kernel methods and their potential use in signal processing

The notion of kernels, recently introduced, has drawn much interest as it allows one to obtain nonlinear algorithms from linear ones in a simple and elegant manner. This, in conjunction with the introduction of new linear classification methods such as the support vector machines (SVMs), has produced significant progress in machine learning and related research topics. The success of such algorithms is now spreading as they are applied to more and more domains. Signal processing procedures can benefit from a kernel perspective, making them more powerful and applicable to nonlinear processing in a simpler and nicer way. We present an overview of kernel methods and provide some guidelines for future development in kernel methods, as well as, some perspectives to the actual signal processing problems in which kernel methods are being applied.     

Quantum classifier with parameterized quantum circuitbased on the isolated quantum system

It is a critical challenge for quantum machine learning to classify the datasets accurately. This article develops a quantum classifier based on the isolated quantum system(QC-IQS) to classify nonlinear and multidimensional datasets. First, a model of QC-IQS is presented by creating parameterized quantum circuits(PQCs) based on the decomposing of unitary operators with the Hamiltonian in the isolated quantum system. Then, a parameterized quantum classification algorithm(QCA) is designed to calcu...     

尺度核函数支撑矢量机

本文提出了一种可容许的支撑矢量机核—尺度核.该尺度核函数可以被看作是一个具有平移因子的多维尺度函数,它能作为平方可积空间的子空间上一组完备的基函数.在此意义上,采用尺度核函数的支撑矢量机,可以认为是在尺度空间中寻找最佳的尺度系数.因此在理论上尺度核函数支撑矢量机能够以零误差逼近某一空间上的任何目标函数,文中给出的仿真实验进一步验证了它的可行性和有效性.     

Classification of hyperspectral remote sensing images with support vector machines

This paper addresses the problem of the classification of hyperspectral remote sensing images by support vector machines (SVMs). First, we propose a theoretical discussion and experimental analysis aimed at understanding and assessing the potentialities of SVM classifiers in hyperdimensional feature spaces. Then, we assess the effectiveness of SVMs with respect to conventional feature-reduction-based approaches and their performances in hypersubspaces of various dimensionalities. To sustain such an analysis, the performances of SVMs are compared with those of two other nonparametric classifiers (i.e., radial basis function neural networks and the K-nearest neighbor classifier). Finally, we study the potentially critical issue of applying binary SVMs to multiclass problems in hyperspectral data. In particular, four different multiclass strategies are analyzed and compared: the one-against-all, the one-against-one, and two hierarchical tree-based strategies. Different performance indicators have been used to support our experimental studies in a detailed and accurate way, i.e., the classification accuracy, the computational time, the stability to parameter setting, and the complexity of the multiclass architecture. The results obtained on a real Airborne Visible/Infrared Imaging Spectroradiometer hyperspectral dataset allow to conclude that, whatever the multiclass strategy adopted, SVMs are a valid and effective alternative to conventional pattern recognition approaches (feature-reduction procedures combined with a classification method) for the classification of hyperspectral remote sensing data.     

Efficient kernel functions for support vector machine regression model for analog circuits’ performance evaluation

Support vector machines (SVMs) have been widely used for creating fast and efficient performance macro-models for quickly predicting the performance parameters of analog circuits. These models have proved to be not only effective and fast but accurate also while predicting the performance. A kernel function is an integral part of SVM to obtain an optimized and accurate model. There is no formal way to decide, which kernel function is suited to a class of regression problem. While most commonly used kernels are radial basis function, polynomial, spline, multilayer perceptron; we have explored many other un-conventional kernel functions and report their efficacy and computational efficiency in this paper. These kernel functions are used with SVM regression models and these macromodels are tested on different analog circuits to check for their robustness and performance. We have used HSPICE for generating the set of learning data. Least Square SVM toolbox along with MATLAB was used for regression. The models which contained modified compositions of kernels were found to be more accurate and thus have lower root mean square error than those containing standard kernels. We have used different CMOS circuits varying in size and complexity as test vehicles—two-stage op amp, cascode op amp, comparator, differential op amp and voltage controlled oscillator.