Drug design by machine learning: support vector machines for pharmaceutical data analysis

We show that the support vector machine (SVM) classification algorithm, a recent development from the machine learning community, proves its potential for structure–activity relationship analysis. In a benchmark test, the SVM is compared to several machine learning techniques currently used in the field. The classification task involves predicting the inhibition of dihydrofolate reductase by pyrimidines, using data obtained from the UCI machine learning repository. Three artificial neural networks, a radial basis function network, and a C5.0 decision tree are all outperformed by the SVM. The SVM is significantly better than all of these, bar a manually capacity-controlled neural network, which takes considerably longer to train.     

A one-layer recurrent neural network for support vector machine learning.

This paper presents a one-layer recurrent neural network for support vector machine (SVM) learning in pattern classification and regression. The SVM learning problem is first converted into an equivalent formulation, and then a one-layer recurrent neural network for SVM learning is proposed. The proposed neural network is guaranteed to obtain the optimal solution of support vector classification and regression. Compared with the existing two-layer neural network for the SVM classification, the proposed neural network has a low complexity for implementation. Moreover, the proposed neural network can converge exponentially to the optimal solution of SVM learning. The rate of the exponential convergence can be made arbitrarily high by simply turning up a scaling parameter. Simulation examples based on benchmark problems are discussed to show the good performance of the proposed neural network for SVM learning.     

基于支持向量机的机械故障智能分类研究

故障样本不足是制约故障诊断技术向智能化方向发展的主要原因之一，支持向量机（SVM)是一种基于统计学习理论(SLT)的机器学习算法，它能在训练样本很少的情况下达到很好的分类效果，从而为故障诊断技术向智能化发展提供了新的途径．本文介绍了支持向量机分类算法，以滚动轴承的故障分类为例，探讨了该算法在故障诊断领域中的应用，并与BP神经网络分类方法进行了对比研究，结果表明，SVM方法在少样本情况下的分类效果优于BP神经网络分类方法．     

Minimal Structure of Self-Organizing HCMAC Neural Network Classifier

The authors previously proposed a self-organizing Hierarchical Cerebellar Model Articulation Controller (HCMAC) neural network containing a hierarchical GCMAC neural network and a self-organizing input space module to solve high-dimensional pattern classification problems. This novel neural network exhibits fast learning, a low memory requirement, automatic memory parameter determination and highly accurate high-dimensional pattern classification. However, the original architecture needs to be hierarchically expanded using a full binary tree topology to solve pattern classification problems according to the dimension of the input vectors. This approach creates many redundant GCMAC nodes when the dimension of the input vectors in the pattern classification problem does not exactly match that in the self-organizing HCMAC neural network. These redundant GCMAC nodes waste memory units and degrade the learning performance of a self-organizing HCMAC neural network. Therefore, this study presents a minimal structure of self-organizing HCMAC (MHCMAC) neural network with the same dimension of input vectors as the pattern classification problem. Additionally, this study compares the learning performance of this novel learning structure with those of the BP neural network,support vector machine (SVM), and original self-organizing HCMAC neural network in terms of ten benchmark pattern classification data sets from the UCI machine learning repository. In particular, the experimental results reveal that the self-organizing MHCMAC neural network handles high-dimensional pattern classification problems better than the BP, SVM or the original self-organizing HCMAC neural network. Moreover, the proposed self-organizing MHCMAC neural network significantly reduces the memory requirement of the original self-organizing HCMAC neural network, and has a high training speed and higher pattern classification accuracy than the original self-organizing HCMAC neural network in most testing benchmark data sets. The experimental results also show that the MHCMAC neural network learns continuous function well and is suitable for Web page classification.     

Unsupervised learning method for a support vector machine and its application to surface electromyogram recognition

The support vector machine (SVM) is known as one of the most influential and powerful tools for solving classification and regression problems, but the original SVM does not have an online learning technique. Therefore, many researchers have introduced online learning techniques to the SVM. In this article, we propose an unsupervised online learning method using a self-organized map for a SVM. Furthermore, the proposed method has a technique for the reconstruction of a SVM. We compare its performance with the original SVM, the supervised learning method for the SVM, and a neural network, and also test our proposed method on surface electromyogram recognition problems.     

面向疾病相关关联抽取的深度语义特征研究

从大量生物医学文献中找出影响疾病的有利因素和有害因素对于疾病的防治研究方向有着重要参考意义。然而,识别疾病影响因素的二分类问题在用传统的机器学习方法进行分类时正确率提升到一定水平后遇到瓶颈难以继续提高。为了提高生物医学领域二分类问题模型的分类性能,利用对于疾病有利和有害的两种因素,采用基于卷积神经网络与支持向量机（SVM）相结合的方法,最终达到超过传统机器学习的性能,使分类的准确率从SVM最佳的90.44%提升到94.38%,从而更好地识别疾病的影响因素。     

基于深度玻尔兹曼机的乐器分类问题研究

应用传统浅层模型处理乐器分类任务存在非线性拟合能力较差的问题,使分类准确率得不到有效保证,有必要引入深度学习方法提升复杂任务的非线性建模能力。将深度玻尔兹曼机作为特征提取器提取表达能力更强的数据特征,分别以SVM与Softmax分类器作为深度神经网络的顶层设置形成DBM SVM组合模型与DBM Softmax组合模型,引入平均场理论和动量项因子优化网络训练过程。将上述两组模型及单一SVM分类器在5类乐器音频数据上进行对比实验,两种深度学习组合模型的分类准确率分别达到89.29%和87.5%,与传统浅层分类方法SVM的73.21%的准确率相比优势明显。实验结果表明深度玻尔兹曼机在乐器分类领域的应用颇具前景。     

A comparative study for content-based dynamic spam classification using four machine learning algorithms

The growth of email users has resulted in the dramatic increasing of the spam emails during the past few years. In this paper, four machine learning algorithms, which are Naïve Bayesian (NB), neural network (NN), support vector machine (SVM) and relevance vector machine (RVM), are proposed for spam classification. An empirical evaluation for them on the benchmark spam filtering corpora is presented. The experiments are performed based on different training set size and extracted feature size. Experimental results show that NN classifier is unsuitable for using alone as a spam rejection tool. Generally, the performances of SVM and RVM classifiers are obviously superior to NB classifier. Compared with SVM, RVM is shown to provide the similar classification result with less relevance vectors and much faster testing time. Despite the slower learning procedure, RVM is more suitable than SVM for spam classification in terms of the applications that require low complexity.     

Fault diagnosis of power transformer based on support vector machine with genetic algorithm

Diagnosis of potential faults concealed inside power transformers is the key of ensuring stable electrical power supply to consumers. Support vector machine (SVM) is a new machine learning method based on the statistical learning theory, which is a powerful tool for solving the problem with small sampling, nonlinearity and high dimension. The selection of SVM parameters has an important influence on the classification accuracy of SVM. However, it is very difficult to select appropriate SVM parameters. In this study, support vector machine with genetic algorithm (SVMG) is applied to fault diagnosis of a power transformer, in which genetic algorithm (GA) is used to select appropriate free parameters of SVM. The experimental data from several electric power companies in China are used to illustrate the performance of the proposed SVMG model. The experimental results indicate that the SVMG method can achieve higher diagnostic accuracy than IEC three ratios, normal SVM classifier and artificial neural network.     

Empirical analysis of support vector machine ensemble classifiers

Ensemble classification – combining the results of a set of base learners – has received much attention in the machine learning community and has demonstrated promising capabilities in improving classification accuracy. Compared with neural network or decision tree ensembles, there is no comprehensive empirical research in support vector machine (SVM) ensembles. To fill this void, this paper analyses and compares SVM ensembles with four different ensemble constructing techniques, namely bagging, AdaBoost, Arc-X4 and a modified AdaBoost. Twenty real-world data sets from the UCI repository are used as benchmarks to evaluate and compare the performance of these SVM ensemble classifiers by their classification accuracy. Different kernel functions and different numbers of base SVM learners are tested in the ensembles. The experimental results show that although SVM ensembles are not always better than a single SVM, the SVM bagged ensemble performs as well or better than other methods with a relatively higher generality, particularly SVMs with a polynomial kernel function. Finally, an industrial case study of gear defect detection is conducted to validate the empirical analysis results.     

基于SVM-MLP的乳腺癌预测

乳腺癌一直是影响女性健康最重要的问题之一,已经成为全球女性发病率最高的恶性肿瘤.近年来,利用机器学习和深度学习方法来诊断癌症已经成为发展较快的一个分支.通过使用逻辑回归模型(LR)、高斯核函数支持向量机(SVM)、前馈神经网络(MLP)对同一数据集进行预测,得出其中SVM迭代时间最短,前馈神经网络预测准确率最高.为了减...     

Solving the stochastic support vector regression with probabilistic constraints by a high-performance neural network model

This paper offers a recurrent neural network to support vector machine (SVM) learning in stochastic support vector regression with probabilistic constraints. The SVM is first converted into an equivalent quadratic programming (QP) formulation in linear and nonlinear cases. An artificial neural network for SVM learning is then proposed. The presented neural network framework guarantees obtaining the optimal solution of the SVM problem. The existence and convergence of the trajectories of the network are studied. The Lyapunov stability for the considered neural network is also shown. The efficiency of the proposed method is shown by three illustrative examples.

     

支持向量机在模式识别中的核函数特性分析

支持向量机是20世纪90年代中期发展起来的一种机器学习技术,与传统人工神经网络不同之处在于前者基于结构风险最小化原理,后者基于经验风险最小化原理。支持向量机不仅结构简单,而且技术性能尤其是泛化能力与BP神经网络相比有明显提高。讨论了支持向量机的分类原理,并用多项式函数、径向基函数和感知机函数等3种核函数作为内积回旋,分别以平面点集分类、手写体汉字识别及双螺旋线识别为例,在不同的结构参数下进行了仿真实验,并对3种核函数的分类特性进行了对比分析,给出了在不同模式识别问题中3种核函数的选择条件。     

Algorithm learning based neural network integrating feature selection and classification

Feature selection and classification techniques have been studied independently without considering the interaction between both procedures, which leads to a degraded performance. In this paper, we present a new neural network approach, which is called an algorithm learning based neural network (ALBNN), to improve classification accuracy by integrating feature selection and classification procedures. In general, a knowledge-based artificial neural network operates on prior knowledge from domain experience, which provides it with better starting points for the target function and leads to better classification accuracy. However, prior knowledge is usually difficult to identify. Instead of using unknown background resources, the proposed method utilizes prior knowledge that is mathematically calculated from the properties of other learning algorithms such as PCA, LARS, C4.5, and SVM. We employ the extreme learning machine in this study to help obtain better initial points faster and avoid irrelevant time-consuming work, such as determining architecture and manual tuning. ALBNN correctly approximates a target hypothesis by both considering the interaction between two procedures and minimizing individual procedure errors. The approach produces new relevant features and improves the classification accuracy. Experimental results exhibit improved performance in various classification problems. ALBNN can be applied to various fields requiring high classification accuracy.     

两种智能交通标志分类器的比较研究

本文介绍了一种可用于交通标志识别的新方法--支持向量机（SVM）算法,并将SVM算法与BP算法在交通标志的粗、细分类中的识别效果进行了对比分析。用中国的116个和日本的23个交通标志标准图分别训练基于SVM算法和基于BP算法的智能分类器,并用中国标志的噪声图、扭曲图和531个日本交通标志实景图作为测试集。在粗分类中,虽然BP算法 法的识别率也能达到90%以上,但SVM算法的识别率几乎可达100%,二者差距明显。在细分类中,SVM算法的识别效果与BP算法相比具有更加明显的优势。实验研究结果表明,SVM算法可以以接近最优的方式解决模式分类问题,同时具有更好的泛化能力,在交通标志识别领域具有良好的研究价值和应用前景。     

Support vector machine with adaptive parameters in financial time series forecasting

A novel type of learning machine called support vector machine (SVM) has been receiving increasing interest in areas ranging from its original application in pattern recognition to other applications such as regression estimation due to its remarkable generalization performance. This paper deals with the application of SVM in financial time series forecasting. The feasibility of applying SVM in financial forecasting is first examined by comparing it with the multilayer back-propagation (BP) neural network and the regularized radial basis function (RBF) neural network. The variability in performance of SVM with respect to the free parameters is investigated experimentally. Adaptive parameters are then proposed by incorporating the nonstationarity of financial time series into SVM. Five real futures contracts collated from the Chicago Mercantile Market are used as the data sets. The simulation shows that among the three methods, SVM outperforms the BP neural network in financial forecasting, and there are comparable generalization performance between SVM and the regularized RBF neural network. Furthermore, the free parameters of SVM have a great effect on the generalization performance. SVM with adaptive parameters can both achieve higher generalization performance and use fewer support vectors than the standard SVM in financial forecasting.     

核函数的选择与改进在人脸识别中的应用

核函数方法广泛应用于人工神经网络和支持向量机等机器学习领域,该方法的采用有效地避免了特征空间中的维数灾难的问题,改善了学习机的分类性能。但是核函数的选择及新的核函数构造一直机器学习领域的核心问题,直接关系到学习机性能的好坏。然而,这个方向的研究成果不多。以支持向量机为例,通过对核矩阵一些特性的计算和研究,从理论上对常用的核函数性能进行了预测。在此基础上,通过实验仿真证实了通过优选后的核函数所组成的混合核函数对分类性能的改善。在加权系数选择合适的情况下,学习机的识别率甚至可以达到100%。所以,不但构造出了性能优异的学习机,而且为核函数的选择提供了参考。     

基于错误驱动算法组合分类器及其在问题分类中的应用

开放领域问答系统(QA)能够给用户提供相对简洁、准确的结果,越来越受到人们的关注.问题分类把问题分成若干语义类型,是QA系统的一个重要的模块,它的准确性直接影响到QA系统的性能.为提高分类器性能,在问题分类任务中使用了集成学习方法,并且实验比较了词汇、句法、同义词集等不同的分类特征及错误驱动、投票法、BP神经网络等分类器集成方法,通过采用基于错误驱动集成分类器,用规则方法TBL作为统计方法SVM的补充;利用来自Wordnet的同义词集和名词的上位概念及Minipar的依存关系等语言知识作为分类特征,在公开测试集中取得了更高的分类精度.     

A Self-Organizing TS-Type Fuzzy Network With Support Vector Learning and its Application to Classification Problems

A self-organizing Takagi-Sugeno (TS)-type fuzzy network with support vector learning (SOTFN-SV) is proposed in this paper. The proposed SOTFN-SV is inspired by analysis of TS-type fuzzy systems and composite-kernel support vector machine (SVM). SOTFN-SV is a fuzzy system constructed by the hybridization of fuzzy clustering and SVM. The antecedent part of SOTFN-SV is generated via fuzzy clustering of the input data, and then SVM is used to tune the consequent part parameters to give the network better generalization performance. For demonstration, SOTFN-SV is applied to several classification problems, especially the skin color classification problem. In the skin color classification application, each color pixel is represented by hue and saturation (HS) color space. To represent color information by histogram as accurately as possible, a nonuniform partition of HS space is proposed. For comparison, SVMs and other fuzzy systems trained by SVM or neural networks are applied to the same classification problems. The advantages of SOTFN-SV are verified by comparisons with the results of these methods.     

几种机器学习方法在人脸识别中的性能比较

BP神经网络、RBF神经网络、支持向量机（SVM）和集成学习是目前应用最为广泛的四种机器学习方法。将这四种常用的机器学习方法分别应用于人脸识别,并利用ORL人脸图像库对各学习方法性能进行了测试和评估。测试结果表明SVM和集成学习在实验中取得了较好的性能,最适合用于人脸识别中特征分类器。