Analog neural network for support vector machine learning

An analog neural network for support vector machine learning is proposed, based on a partially dual formulation of the quadratic programming problem. It results in a simpler circuit implementation with respect to existing neural solutions for the same application. The effectiveness of the proposed network is shown through some computer simulations concerning benchmark problems     

A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming.

In this paper, a one-layer recurrent neural network with a discontinuous hard-limiting activation function is proposed for quadratic programming. This neural network is capable of solving a large class of quadratic programming problems. The state variables of the neural network are proven to be globally stable and the output variables are proven to be convergent to optimal solutions as long as the objective function is strictly convex on a set defined by the equality constraints. In addition, a sequential quadratic programming approach based on the proposed recurrent neural network is developed for general nonlinear programming. Simulation results on numerical examples and support vector machine (SVM) learning show the effectiveness and performance of the neural network.     

Approximating support vector machine with artificial neural network for fast prediction

Support vector machine (SVM) is a powerful algorithm for classification and regression problems and is widely applied to real-world applications. However, its high computational load in the test phase makes it difficult to use in practice. In this paper, we propose hybrid neural network (HNN), a method to accelerate an SVM in the test phase by approximating the SVM. The proposed method approximates the SVM using an artificial neural network (ANN). The resulting regression function of the ANN replaces the decision function or the regression function of the SVM. Since the prediction of the ANN requires significantly less computation than that of the SVM, the proposed method yields faster test speed. The proposed method is evaluated by experiments on real-world benchmark datasets. Experimental results show that the proposed method successfully accelerates SVM in the test phase with little or no prediction loss.     

Fast-forward solver for inhomogeneous media using machine learning methods: artificial neural network,support vector machine and fuzzy logic

Encountering with a nonlinear second-order differential equation including ϵ _r and μ _r spatial distributions, while computing the fields inside inhomogeneous media, persuaded us to find their known distributions that give exact solutions. Similarities between random distributions of electric properties and known functions lead us to estimate them using three mathematical tools of artificial neural networks (ANNs), support vector machines (SVMs) and Fuzzy Logic (FL). Assigning known functions after fitting with minimum error to arbitrary inputs using results of machine learning networks leads to achieve an approximate solution for the field inside materials considering boundary conditions. A comparative study between the methods according to the complexity of the structures as well as the accuracy and the calculation time for testing of unforeseen inputs, including classification, prediction and regression is presented. We examined the extracted pairs of ϵ _r and μ _r with ANN, SVM networks and FL and got satisfactory outputs with detailed results. The application of the presented method in zero reflection subjects is exemplified.

     

一种改进的支持向量机增量学习算法

提出了一种改进的支持向量机增量学习算法。分析了新样本加入后,原样本和新样本中哪些样本可能转化为新支持向量。基于分析结论提出了一种改进的学习算法。该算法舍弃了对最终分类无用的样本,并保留了有用的样本。对标准数据集的实验结果表明,该算法在保证分类准确度的同时大大减少了训练时间。     

支持向量机与神经网络相结合的板带凸度预测

为提高热轧生产过程中板带凸度的预测精度,提出了一种将粒子群优化算法(particle swarm optimization, PSO)、支持向量回归(support vector regression, SVR)和BP神经网络(back propagation neural network, BPNN)相结合的板带凸度预测模型。采用PSO算法优化SVR模型的参数,建立了PSO-SVR板带凸度预测模型,提出采用BPNN建立板带凸度偏差模型与PSO-SVR板带凸度模型相结合的方法对板带凸度进行预测。采用现场数据对模型的预测精度进行验证,并采用统计指标评价模型的综合性能。仿真结果表明,与PSO-SVR、SVR、BPNN和GA-SVR模型进行比较,PSO-SVR+BPNN模型具有较高的学习能力和泛化能力,并且比GA-SVR模型运算时间短。     

支持向量机与BP网络在火灾图像探测上的比较

针对BP神经网络和支持向量机在火灾探测上存在的理论差别,分别构建了基于此2种方法的火灾图像探测方法.2种方法均依据火焰颜色分布规律实现了目标区域的分离,并将目标区域的形状特征及变化值作为判据.通过对火灾实验样本的训练及识别2,种方法的探测表现得到了比较与分析.实验结果表明基于支持向量机的火灾探测方法具有快速收敛特性及所需较少训练样本的优点.同时,BP神经网络对测试集较少的错判反映出其良好的非线性映射能力,适合求解内部机制复杂的问题.     

支持向量机与RBF神经网络回归性能比较研究

支持向量机与RBF神经网络相比各有优缺点,通过对支持向量机与RBF神经网络的研究,从理论上分析了这两种学习机在回归预测原理上的异同,通过仿真实验对比了两者在测试集上的逼近能力及泛化能力。仿真结果表明,对于小样本集,支持向量机的逼近能力及泛化能力要优于RBF神经网络。对实际应用中回归模型的选择问题提出了建议。     

A machine learning method for extracting symbolic knowledge from recurrent neural networks

Neural networks do not readily provide an explanation of the knowledge stored in their weights as part of their information processing. Until recently, neural networks were considered to be black boxes, with the knowledge stored in their weights not readily accessible. Since then, research has resulted in a number of algorithms for extracting knowledge in symbolic form from trained neural networks. This article addresses the extraction of knowledge in symbolic form from recurrent neural networks trained to behave like deterministic finite-state automata (DFAs). To date, methods used to extract knowledge from such networks have relied on the hypothesis that networks' states tend to cluster and that clusters of network states correspond to DFA states. The computational complexity of such a cluster analysis has led to heuristics that either limit the number of clusters that may form during training or limit the exploration of the space of hidden recurrent state neurons. These limitations, while necessary, may lead to decreased fidelity, in which the extracted knowledge may not model the true behavior of a trained network, perhaps not even for the training set. The method proposed here uses a polynomial time, symbolic learning algorithm to infer DFAs solely from the observation of a trained network's input-output behavior. Thus, this method has the potential to increase the fidelity of the extracted knowledge.     

Inductive manifold learning using structured support vector machine

Most manifold learning techniques are used to transform high-dimensional data sets into low-dimensional space. In the use of such techniques, after unseen data samples are added to the data set, retraining is usually necessary. However, retraining is a time-consuming process and no guarantee of the transformation into the exactly same coordinates, thus presenting a barrier to the application of manifold learning as a preprocessing step in predictive modeling. To solve this problem, learning a mapping from high-dimensional representations to low-dimensional coordinates is proposed via structured support vector machine. After training a mapping, low-dimensional representations of unobserved data samples can be easily predicted. Experiments on several datasets show that the proposed method outperforms the existing out-of-sample extension methods.     

Kernel-based online machine learning and support vector reduction

We apply kernel-based machine learning methods to online learning situations, and look at the related requirement of reducing the complexity of the learnt classifier. Online methods are particularly useful in situations which involve streaming data, such as medical or financial applications. We show that the concept of span of support vectors can be used to build a classifier that performs reasonably well while satisfying given space and time constraints, thus making it potentially suitable for such online situations. The span-based heuristic is observed to be effective under stringent memory limits (that is when the number of support vectors a machine can hold is very small).     

Online learning method using support vector machine for surface-electromyogram recognition

Research surface electromyogram (s-EMG) signal recognition using neural networks is a method which identifies the relation between s-EMG patterns. However, it is not sufficiently satisfying for the user because s-EMG signals change according to muscle wasting or to changes in the electrode position, etc. A support vector machine (SVM) is one of the most powerful tools for solving classification problems, but it does not have an online learning technique. In this article, we propose an online learning method using SVM with a pairwise coupling technique for s-EMG recognition. We compared its performance with the original SVM and a neural network. Simulation results showed that our proposed method is better than the original SVM. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

基于LS_SVM的不确定系统神经滑模控制方法研究

针对一类参数大范围变化的不确定系统,提出一种基于分类转换策略的神经滑模控制方法．按小偏差原理对系统模型进行划分,利用结合主成分分析的最小二乘支持向量机进行分类训练,并分别设计基于径向基函数神经网络在线调整切换项增益的滑模控制器,在线时利用分类器按系统数据自动选择相应的控制器．同时,引入结合混沌机制的量子粒子群算法,并将其用于控制器近似最佳切换函数的构造．仿真结果表明,系统具有良好的跟踪性能和较强的鲁棒性,有效地降低了抖振．     

An online incremental learning support vector machine for large-scale data

Support Vector Machines (SVMs) have gained outstanding generalization in many fields. However, standard SVM and most of modified SVMs are in essence batch learning, which make them unable to handle incremental learning or online learning well. Also, such SVMs are not able to handle large-scale data effectively because they are costly in terms of memory and computing consumption. In some situations, plenty of Support Vectors (SVs) are produced, which generally means a long testing time. In this paper, we propose an online incremental learning SVM for large data sets. The proposed method mainly consists of two components: the learning prototypes (LPs) and the learning Support Vectors (LSVs). LPs learn the prototypes and continuously adjust prototypes to the data concept. LSVs are to get a new SVM by combining learned prototypes with trained SVs. The proposed method has been compared with other popular SVM algorithms and experimental results demonstrate that the proposed algorithm is effective for incremental learning problems and large-scale problems.     

Web-based algorithm for cylindricity evaluation using support vector machine learning

This paper introduces a cylindricity evaluation algorithm based on support vector machine learning with a specific kernel function, referred to as SVR, as a viable alternative to traditional least square method (LSQ) and non-linear programming algorithm (NLP). Using the theory of support vector machine regression, the proposed algorithm in this paper provides more robust evaluation in terms of CPU time and accuracy than NLP and this is supported by computational experiments. Interestingly, it has been shown that the SVR significantly outperforms LSQ in terms of the accuracy while it can evaluate the cylindricity in a more robust fashion than NLP when the variance of the data points increases. The robust nature of the proposed algorithm is expected because it converts the original nonlinear problem with nonlinear constraints into other nonlinear problem with linear constraints. In addition, the proposed algorithm is programmed using Java Runtime Environment to provide users with a Web based open source environment. In a real-world setting, this would provide manufacturers with an algorithm that can be trusted to give the correct answer rather than making a good part rejected because of inaccurate computational results.     

Soft multimedia anomaly detection based on neural network and optimization driven support vector machine

Software multimedia anomaly detection model based on neural network and optimization driven support vector machine is discussed in this paper. For multimedia information, most traditional information security technology has its limitations. For example, the limitation of the encryption technology is that on the one hand, the encrypted files resulting from the incomprehension of attributes interfere with the transfer of multimedia information. On the other hand, the encrypted multimedia information is likely to attract the attacker’s curiosity and attention, and is likely to be cracked, and once it is cracked, the system loses control of the information. To deal with these challenges, this study integrates soft computing techniques to finalize the enhanced multimedia anomaly detection model. With respect to the neural network, a random system with random factors is referred to as a random system. These practical systems are generally described and modeled by stochastic differential equations. In this study, we combined the double support vector machine and decision tree support vector machine to construct a new double support vector machine decision tree classifier. Kernel function and convex optimization were integrated to guarantee an optimal solution. Experimental results demonstrated the robustness of the model compared with other recent techniques.

     

基于可拓聚类的极限学习机神经网络

针对极限学习机(ELM)算法随机选择输入层权值的问题,借鉴第2类型可拓神经网络(ENN-2)聚类的思想,提出了一种基于可拓聚类的ELM(EC-ELM)神经网络。该神经网络是以隐含层神经元的径向基中心向量作为输入层权值,采用可拓聚类算法动态调整隐含层节点数目和径向基中心,并根据所确定的输入层权值,利用Moore-Penrose广义逆快速完成输出层权值的求解。同时,对标准的Friedman#1回归数据集和Wine分类数据集进行测试,结果表明,EC-ELM提供了一种简便的神经网络结构和参数学习方法,并且比基于可拓理论的径向基函数(ERBF)、ELM神经网络具有更高的建模精度和更快的学习速度,为复杂过程的建模提供了新思路。     

A flexible support vector machine for regression

In this paper, a novel regression algorithm coined flexible support vector regression is proposed. We first model the insensitive zone in classic support vector regression, respectively, by its up- and down-bound functions and then give a kind of generalized parametric insensitive loss function (GPILF). Subsequently, based on GPILF, we propose an optimization criterion such that the unknown regressor and its up- and down-bound functions can be found simultaneously by solving a single quadratic programming problem. Experimental results on both several publicly available benchmark data sets and time series prediction show the feasibility and effectiveness of the proposed method.     

适用于不确定类标签数据学习的迁移支持向量机

为了解决包含不确定信息的分类学习问题,提出一种新的适用于不确定类标签数据的迁移支持向量机。该方法基于结构风险最小化模型,同时将源领域中所学知识、领域间的共享数据、目标领域中已标定的和不确定的数据纳入学习框架中,进而实现了源领域和目标领域的知识迁移。在多种真实数据集上的实验结果表明了所提出方法的有效性。