一类非线性系统基于最小二乘支持向量机的直接自适应控制

提出一类非线性系统基于最小二乘支持向量机的直接自适应控制方法.该方法采用最小二乘支持向量机构造自适应控制器,自适应控制器参数的在线调整规律由Lyapunov稳定性理论导出,并严格证明了闭环系统的渐近稳定性.仿真研究表明了此控制方案的可行性和有效性.     

Subspace identification of Hammerstein systems using least squares support vector machines

This paper presents a method for the identification of multiple-input-multiple-output (MIMO) Hammerstein systems for the goal of prediction. The method extends the numerical algorithms for subspace state space system identification (N4SID), mainly by rewriting the oblique projection in the N4SID algorithm as a set of componentwise least squares support vector machines (LS-SVMs) regression problems. The linear model and static nonlinearities follow from a low-rank approximation of a matrix obtained from this regression problem.     

Impulse response constrained LS-SVM modelling for MIMO Hammerstein system identification

In this paper, a new methodology for identifying multiple inputs multiple outputs Hammerstein systems is presented. The proposed method aims at incorporating the impulse response of the system into a least-squares support vector machine (LS-SVM) formulation and therefore the regularisation capabilities of LS-SVM are applied to the system as a whole. One of the main advantages of this method comes from the fact that it is flexible concerning the class of problems it can model and that no previous knowledge about the underlying non-linearities is required except for very mild assumptions. Also, it naturally adapts to handle different numbers of inputs/outputs and performs well in the presence of white Gaussian noise. Finally, the method incorporates information about the structure of the system but still the solution of the model follows from a linear system of equations. The performance of the proposed methodology is shown through three simulation examples and compared with other methods in the literature.     

基于LS-SVM的位置指纹室内定位

基于无线接入点（Access Point,AP）接收信号强度（Received Signal Strength,RSS）的位置指纹室内定位技术近几年已经成为国内外位置感知研究的热点。提出了基于最小二乘支持向量机（Least Squares Support Vector Machines,LS-SVM）的位置指纹定位方法。给出了基于LS-SVM的指纹定位模型,描述了LS-SVM指纹样本训练的具体实现过程。重点在于将定位问题转化为一个多类别分类问题,并分别采用一对一（OAO）和一对多（OAA）方法将其转化为多个二值分类问题。仿真结果表明,LS-SVM较传统支持向量机（SVMs）、K近邻（k-Nearest Neighbors,K-NN）定位方法的分类准确率高且计算代价小,平均分类准确率达92.00%。     

Identification of MIMO Hammerstein systems using cardinal spline functions

A new approach to identify multivariable Hammerstein systems is proposed in this paper. By using cardinal cubic spline functions to model the static nonlinearities, the proposed method is effective in modelling processes with hard and/or coupled nonlinearities. With an appropriate transformation, the nonlinear models are parameterized such that the nonlinear identification problem is converted into a linear one. The persistently exciting condition for the transformed input is derived to ensure the estimates are consistent with the true system. A simulation study is performed to demonstrate the effectiveness of the proposed method compared with the existing approaches based on polynomials.     

动态贝叶斯最小二乘支持向量机

为了对涡扇发动机的运行参数变化进行实时高精度预测,提出一种基于动态贝叶斯最小二乘支持向量机(LS-SVM)的时间序列预测算法。该算法将贝叶斯证据框架理论用于推断LS-SVM的初始模型参数;然后,利用样本增减迭代学习算法实现LS-SVM的参数动态调整。对某型涡扇发动机的摩擦力矩时间序列进行动态预测,并与动态LS-SVM模型的预测结果进行比较。结果显示,动态贝叶斯LS-SVM具有较好的预测精度。     

Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression

Given n training examples, the training of a least squares support vector machine (LS-SVM) or kernel ridge regression (KRR) corresponds to solving a linear system of dimension n. In cross-validating LS-SVM or KRR, the training examples are split into two distinct subsets for a number of times (l) wherein a subset of m examples are used for validation and the other subset of (n-m) examples are used for training the classifier. In this case l linear systems of dimension (n-m) need to be solved. We propose a novel method for cross-validation (CV) of LS-SVM or KRR in which instead of solving l linear systems of dimension (n-m), we compute the inverse of an n dimensional square matrix and solve l linear systems of dimension m, thereby reducing the complexity when l is large and/or m is small. Typical multi-fold, leave-one-out cross-validation (LOO-CV) and leave-many-out cross-validations are considered. For five-fold CV used in practice with five repetitions over randomly drawn slices, the proposed algorithm is approximately four times as efficient as the naive implementation. For large data sets, we propose to evaluate the CV approximately by applying the well-known incomplete Cholesky decomposition technique and the complexity of these approximate algorithms will scale linearly on the data size if the rank of the associated kernel matrix is much smaller than n. Simulations are provided to demonstrate the performance of LS-SVM and the efficiency of the proposed algorithm with comparisons to the naive and some existent implementations of multi-fold and LOO-CV.     

大型呼叫中心人工呼入量的最小二乘支持向量机模型

通过分析大型呼叫中心人工呼入量的数据特点, 文中将呼入量分解为日呼入量与相应时间段呼入量, 利用最小二乘支持向量机(LS-SVM)的原理, 建立日呼入量与时间段呼入量两个时间序列预测模型. 实验仿真证明, 采用该方法建立的日呼入量与时间段呼入量预测模型, 在回归和预测方面都可以得到满意的结果. 通过与神经网络预测模型的对比分析, LS-SVM总体上优于人工神经网络的预测效果.     

1-Norm least squares twin support vector machines

During the last few years, nonparallel plane classifiers, such as Multisurface Proximal Support Vector Machine via Generalized Eigenvalues (GEPSVM), and Least Squares TWSVM (LSTSVM), have attracted much attention. However, there are not any modifications of them that have been presented to automatically select the input features. This motivates the rush towards new classifiers. In this paper, we develop a new nonparallel plane classifier, which is designed for automatically selecting the relevant features. We first introduce a Tikhonov regularization (TR) term that is usually used for regularizing least squares into the LSTSVM learning framework, and then convert this formulation to a linear programming (LP) problem. By minimizing an exterior penalty (EP) problem of the dual of the LP formulation and using a fast generalized Newton algorithm, our method yields very sparse solutions, such that it generates a classifier that depends on only a smaller number of input features. In other words, this approach is capable of suppressing input features. This makes the classifier easier to store and faster to compute in the classification phase. Lastly, experiments on both toy and real problems disclose the effectiveness of our method.     

Nolinear model predictive control using Hammerstein models

Nonlinear models that are composed of a linear dynamic element in series with a nonlinear static element prove to be very attractive in describing the behaviour of many chemical processes. In this paper, a model predictive control scheme is proposed using the Hammerstein model structure. Two simulation examples, a pH neutralization process and a binary distillation column, are used to demonstrate the effectiveness of the method.     

A global convergent outlier robust adaptive predictor for MIMO Hammerstein models

The paper considers the outlier‐robust recursive stochastic approximation algorithm for adaptive prediction of multiple‐input multiple‐output (MIMO) Hammerstein model with a static nonlinear block in polynomial form and a linear block is output error (OE) model. It is assumed that there is a priori information about a distribution class to which a real disturbance belongs. Within the framework of these assumptions, the main contributions of this paper are: (i) for MIMO Hammerstein OE model, the stochastic approximation algorithm, based on robust statistics (in the sense of Huber), is derived; (ii) scalar gain of algorithm is exactly determined using the Laplace function; and (iii) a global convergence of robust adaptive predictor is proved. The proof is based on martingale theory and generalized strictly positive real conditions. Practical behavior of algorithm was illustrated by simulations. Copyright © 2016 John Wiley & Sons, Ltd.     

支持向量机和最小二乘支持向量机的比较及应用研究

介绍和比较了支持向量机分类器和量小二乘支持向量机分类器的算法。并将支持向量机分类器和量小二乘支持向量机分类器应用于心脏病诊断，取得了较高的准确率。所用数据来自UCI bench—mark数据集。实验结果表明，支持向量机和量小二乘支持向量机在医疗诊断中有很大的应用潜力。     

最小二乘支持向量机在睡眠打鼾诊断中的应用

支持向量机是数据挖掘和机器学习领域中的重要方法之一,最小二乘支持向量机是支持向量机学习算法的重要扩展,在训练速度方面有明显优势。对支持向量机现有的多类分类算法（一对一方法、一对多方法、纠错输出编码方法和最小输出编码方法）引入了最小二乘支持向量机,并应用于睡眠打鼾疾病的诊断预测中,取得了较好的效果。     

Prediction of flashover voltage of insulators using least squares support vector machines

The importance of the research on insulator pollution has been increased considerably with the rise of the voltage of transmission lines. In order to determine the flashover behavior of polluted high voltage insulators and to identify to physical mechanisms that govern this phenomenon, the researchers have been brought to establish a modeling. In this paper, a dynamic model of AC flashover voltages of the polluted insulators is constructed using the least square support vector machine (LS-SVM) regression method. For this purpose, a training set is generated by using a numerical method based on Finite Element Method (FEM) for several of common insulators with different geometries. To improve the resulting model’s generalization ability, an efficient optimization algorithm known as the grid search are adopted to tune parameters in LS-SVM design.In addition, two different testing set, which are not introduced to the LS-SVM during the training procedures, is used to evaluate the effectiveness and feasibility of the proposed method. Then, optimum LS-SVM model is firstly obtained and the performance of the proposed system with other intelligence method based on ANN is compared. It can be concluded that the performance of LS-SVM model outperforms those of ANN, for the data set available, which indicates that the LS-SVM model has better generalization ability.     

基于AdaBoost算法的加权二乘向量回归机

针对二乘向量机(LS-SVM)对所有样本误差惩罚相同、预测精度不高的问题,提出了一种基于AdaBoost模型的二乘向量回归机。该算法使用多个二乘向量机按照某种学习规则协调各二乘向量机的输出,同时根据回归精度,建立各二乘向量机中每一个样本的误差惩罚权重,以突出样本的惩罚差异性,提高算法的泛化性能。实验结果表明,提出的算法提高了二乘向量回归机的预测精度,优化了学习机的性能。     

Soft clustering using weighted one-class support vector machines

This paper describes a new soft clustering algorithm in which each cluster is modelled by a one-class support vector machine (OC-SVM). The proposed algorithm extends a previously proposed hard clustering algorithm, also based on OC-SVM representation of clusters. The key building block of our method is the weighted OC-SVM (WOC-SVM), a novel tool introduced in this paper, based on which an expectation-maximization-type soft clustering algorithm is defined. A deterministic annealing version of the algorithm is also introduced, and shown to improve the robustness with respect to initialization. Experimental results show that the proposed soft clustering algorithm outperforms its hard clustering counterpart, namely in terms of robustness with respect to initialization, as well as several other state-of-the-art methods.     

Temperature prediction control based on least squares support vector machines

A prediction control algorithm is presented based on least squares support vector machines (LS-SVM) model for a class of complex systems with strong nonlinearity. The nonlinear off-line model of the controUed plant is built by LS-SVM with radial basis function (RBF) kernel. In the process of system running, the off-line model is linearized at each sampling instant, and the generalized prediction control (GPC) algorithm is employed to implement the prediction control for the controlled plant. The obtained algorithm is applied to a boiler temperature control system with complicated nonlinearity and large time delay. The results of the experiment verify the effectiveness and merit of the algorithm.     

Knowledge discovery in a direct marketing case using least squares support vector machines

We study the problem of repeat‐purchase modeling in a direct marketing setting using Belgian data. More specifically, we investigate the detection and qualification of the most relevant explanatory variables for predicting purchase incidence. The analysis is based on a wrapped form of input selection using a sensitivity based pruning heuristic to guide a greedy, stepwise, and backward traversal of the input space. For this purpose, we make use of a powerful and promising least squares support vector machine (LS‐SVM) classifier formulation. This study extends beyond the standard recency frequency monetary (RFM) modeling semantics in two ways: (1) by including alternative operationalizations of the RFM variables, and (2) by adding several other (non‐RFM) predictors. Results indicate that elimination of redundant/irrelevant inputs allows significant reduction of model complexity. The empirical findings also highlight the importance of frequency and monetary variables, while the recency variable category seems to be of somewhat lesser importance to the case at hand. Results also point to the added value of including non‐RFM variables for improving customer profiling. More specifically, customer/company interaction, measured using indicators of information requests and complaints, and merchandise returns provide additional predictive power to purchase incidence modeling for database marketing. © 2001 John Wiley & Sons, Inc.     

一种改进的在线最小二乘支持向量机回归算法

针对一般最小二乘支持向量机处理大规模数据集会出现训练速度幔、计算量大、不易在线训练的缺点,将修正后的遗忘因子矩形窗方法与支持向量机相结合,提出一种基于改进的遗忘因子矩形窗算法的在线最小二乘支持向量机回归算法,既突出了当前窗口数据的作用,又考虑了历史数据的影响.所提出的算法可减少计算量,提高在线辨识精度.仿真算例表明了该方法的有效性.     

Improved conjugate gradient implementation for least squares support vector machines

As a promising method for pattern recognition and function estimation, least squares support vector machines (LS-SVM) express the training in terms of solving a linear system instead of a quadratic programming problem as for conventional support vector machines (SVM). In this paper, by using the information provided by the equality constraint, we transform the minimization problem with a single equality constraint in LS-SVM into an unconstrained minimization problem, then propose reduced formulations for LS-SVM. By introducing this transformation, the times of using conjugate gradient (CG) method, which is a greatly time-consuming step in obtaining the numerical solution, are reduced to one instead of two as proposed by Suykens et al. (1999). The comparison on computational speed of our method with the CG method proposed by Suykens et al. and the first order and second order SMO methods on several benchmark data sets shows a reduction of training time by up to 44%.