Linear programming support vector regression with wavelet kernel: A new approach to nonlinear dynamical systems identification

Wavelet theory has a profound impact on signal processing as it offers a rigorous mathematical framework to the treatment of multiresolution problems. The combination of soft computing and wavelet theory has led to a number of new techniques. On the other hand, as a new generation of learning algorithms, support vector regression (SVR) was developed by Vapnik et al. recently, in which ?-insensitive loss function was defined as a trade-off between the robust loss function of Huber and one that enables sparsity within the SVs. The use of support vector kernel expansion also provides us a potential avenue to represent nonlinear dynamical systems and underpin advanced analysis. However, for the support vector regression with the standard quadratic programming technique, the implementation is computationally expensive and sufficient model sparsity cannot be guaranteed. In this article, from the perspective of model sparsity, the linear programming support vector regression (LP-SVR) with wavelet kernel was proposed, and the connection between LP-SVR with wavelet kernel and wavelet networks was analyzed. In particular, the potential of the LP-SVR for nonlinear dynamical system identification was investigated.     

LSSVR ensemble learning with uncertain parameters for crude oil price forecasting

Least squares support vector regression (LSSVR) is an effective and competitive approach for crude oil price prediction, but its performance suffers from parameter sensitivity and long tuning time. This paper considers the user-defined parameters as uncertain (or random) factors to construct an LSSVR ensemble learning paradigm, by taking four major steps. First, probability distributions of the user-defined parameters in LSSVR are designed using grid method for low upper bound estimation (LUBE). Second, random sets of parameters are generated according to the designed probability distributions to formulate diverse individual LSSVR members. Third, each individual member is applied to individual prediction. Finally, all individual results are combined to the final output via ensemble weighted averaging, with probabilities measuring the corresponding weights. The computational experiment using the crude oil spot price of West Texas Intermediate (WTI) verifies the effectiveness of the proposed LSSVR ensemble learning paradigm with uncertain parameters compared with some existing LSSVR variants (using other popular parameters selection algorithms), in terms of prediction accuracy and time-saving.     

A random least-trimmed-squares identification algorithm

The Least-trimmed-squares (LTS) estimator is a well known robust estimator in terms of protecting the estimate from the outliers. Its high computational complexity is however a problem in practice. In this paper, we propose a random LTS algorithm which has a low computational complexity that can be calculated a priori as a function of the required error bound and the confidence interval. Moreover, if the number of data points goes to infinite, the algorithm becomes a deterministic one that converges to the true LTS in some probability sense.     

基于改进KPCA与混合核函数LSSVR的泥石流预测

针对引发泥石流灾害的多重影响因素而导致的预测维数灾难，以及最小二乘支持向量回归（least squares support vector regression， LSSVR）模型中选取单核函数而导致的模型训练性能部分缺陷的问题，提出了一种基于改进的核主成分分析（kernel principal component analysis， KPCA）与混合核函数LSSVR的泥石流灾害预测方法.首先，将影响泥石流发生的7种初始因子赋予权重，利用加权KPCA法筛选出3个主成分影响因子作为模型输入；然后，将局部核函数与全局核函数相结合，运用到LSSVR模型上，进行泥石流发生概率预测，以平衡样本学习能力与泛化能力，并使用果蝇优化算法（fruit fly optimization algorithm， FOA）更新模型的最优参数；最后，以磨子沟监测数据进行仿真验证.结果表明，该方法能够有效地降低维数灾难并提升预测模型精确度，在误差允许范围内预测出泥石流发生概率值及对应的预警等级，为相关决策部门提供一定的借鉴经验.     

基于- 范数约束的LSSVR 多核学习算法

针对核函数选择对最小二乘支持向量机回归模型泛化性的影响, 提出一种新的基于????- 范数约束的最小二乘支持向量机多核学习算法. 该算法提供了两种求解方法, 均通过两重循环进行求解, 外循环用于更新核函数的权值, 内循环用于求解最小二乘支持向量机的拉格朗日乘数, 充分利用该多核学习算法, 有效提高了最小二乘支持向量机的泛化能力, 而且对惩罚参数的选择具有较强的鲁棒性. 基于单变量和多变量函数的仿真实验表明了所提出算法的有效性.

     

Outliers, inliers and the generalized least trinuned squares estimator in system identification

The least trimmed squares estimator (LTS) is a well known robust estimator in terms of protecting the estimate from the outliers. Its high computational complexity is however a problem in practice. We show that the LTS estimate can be obtained by a simple algorithm with the complexity O( N In N) for large N, where N is the number of measurements. We also show that though the LTS is robust in terms of the outliers, it is sensitive to the inliers. The concept of the inliers is introduced. Moreover, the Generalized Least Trimmed Squares estimator (GLTS) together with its solution are presented that reduces the effect of both the outliers and the inliers.     

Adaptive online prediction method based on LS-SVR and its application in an electronic system

Health trend prediction has become an effective way to ensure the safe operation of highly reliable systems,and online prediction is always necessary in many real applications.To simultaneously obtain better or acceptable online prediction accuracy and shorter computing time,we propose a new adaptive online method based on least squares support vector regression(LS-SVR).This method adopts two approaches.One approach is that we delete certain support vectors by judging the linear correlation among the samples to increase the sparseness of the prediction model.This approach can control the loss of useful information in sample data,improve the generalization capability of the prediction model,and reduce the prediction time.The other approach is that we reduce the number of traditional LS-SVR parameters and establish a modified simple prediction model.This approach can reduce the calculation time in the process of adaptive online training.Simulation and a certain electric system application indicate preliminarily that the proposed method is an effective prediction approach for its good prediction accuracy and low computing time.     

一种改进的在线最小二乘支持向量机回归算法

针对一般最小二乘支持向量机处理大规模数据集会出现训练速度幔、计算量大、不易在线训练的缺点,将修正后的遗忘因子矩形窗方法与支持向量机相结合,提出一种基于改进的遗忘因子矩形窗算法的在线最小二乘支持向量机回归算法,既突出了当前窗口数据的作用,又考虑了历史数据的影响.所提出的算法可减少计算量,提高在线辨识精度.仿真算例表明了该方法的有效性.     

Outliers, inliers and the generalized least trimmed squares estimator in system identification

The least trimmed squares estimator (LTS) is a well known robust estinaator in terms of protecting the estimatefrom the outliers. Its high computational complexity is however a problem in practice. We show that the LTS estimate can be obtained by a simple algorithm with the complexity O( N In N) for large N, where N is the number of measurements. We also showthat though the LTS is robust in terms of the outliers, it is sensitive to the inliers. The concept of the inliers is introduced. Moreover, the Generalized Least Trimmed Squares estimator (GLTS) together with its solution are presented that reduces the effect of both the outliers and the inliers.     

一种快速稀疏最小二乘支持向量回归机

将Jiao法直接应用于最小二乘支持向量回归机上的效果并不理想,为此采用不完全抛弃的策略,提出了改进的Jiao法,并将其应用于最小二乘支持向量回归机.数据集测试的结果表明,基于改进Jiao法的稀疏最小二乘支持向量回归机,无论在支持向量个数和训练时间上都取得了一定的优势.与其他剪枝算法相比,在不丧失回归精度的情况下,改进的Jiao法可大大缩短训练时间.另外,改进的Jiao法同样适用于分类问题.     

基于LSSVR和LSTM的多模型优化集成负荷预测

针对负荷需求受多源因素影响和现有单模型预测方法精度较低的问题,提出了一种基于最小二乘支持向量回归(LSSVR)和长短期记忆循环神经网络(LSTM)的多模型优化集成负荷预测方法。首先探究负荷相关特征的特性并由互信息进行特征选择,获取最优特征集。在此基础上采用随机抽样(bootstrap)生成多个训练集,然后使用具有良好预测能力的LSSVR和LSTM模型对多个训练集分别进行预测。利用混沌粒子群优化算法(CPSO)进一步提高模型预测精度。最后,在决策阶段中使用偏最小二乘回归(PLSR)组合各个子模型的最优预测输出并提供最终预测结果。对真实电网数据进行了仿真,并与其它预测方法进行了比较。本文所提方法的应用范围广泛且预测精度提高显著。     

Integrated soft sensor using just-in-time support vector regression and probabilistic analysis for quality prediction of multi-grade processes

Multi-grade processes have played an important role in the fine chemical and polymer industries. An integrated nonlinear soft sensor modeling method is proposed for online quality prediction of multi-grade processes. Several single least squares support vector regression (LSSVR) models are first built for each product grade. For online prediction of a new sample, a probabilistic analysis approach using the statistical property of steady-state grades is presented. The prediction can then be obtained using the corresponding LSSVR model if its probability of the special steady-state grade is large enough. Otherwise, the query sample is considered located in the transitional mode because it is not similar to any steady-state grade. In this situation, a just-in-time LSSVR (JLSSVR) model is constructed using the most similar samples around it. To improve the efficiency of searching for similar samples of JLSSVR, a strategy combined with the characteristics of multi-grade processes is proposed. Additionally, the similarity factor and similar samples of JLSSVR can be determined adaptively using a fast cross-validation strategy with low computational load. The superiority of the proposed soft sensor is first demonstrated through a simulation example. It is also compared with other soft sensors in terms of online prediction of melt index in an industrial plant in Taiwan.     

时变过程在线辨识的即时递推核学习方法研究

为了及时跟踪非线性化工过程的时变特性, 提出即时递推核学习 (Kernel learning, KL)的在线辨识方法. 针对待预测的新样本点, 采用即时学习 (Just-in-time kernel learning, JITL)策略, 通过构造累积相似度因子, 选择与其相似的样本集建立核学习辨识模型. 为避免传统即时学习对每个待预测点都重新建模的繁琐, 利用两个临近时刻相似样本集的异同点, 采用递推方法有效添加新样本, 并删减旧模型的样本, 以快速建立新即时模型. 通过一时变连续搅拌釜式反应过程的在线辨识, 表明了所提出方法在保证计算效率的同时, 较传统递推核学习方法提高了辨识的准确程度, 能更好地辨识时变过程.     

一种模拟电路的支持向量机故障诊断方法

系统地提出了模拟电路的最小二乘小波支持向量机故障诊断方法。从测试点得到各种故障状态下的输出电压信号，对输出电压信号进行小波去噪，对信号进行小波分解获取多尺度的低频系数和高频系数，并对小波系数进行处理从而提取出故障特征量，以此作为学习样本来训练最小二乘小波支持向量机，确定其模拟电路故障诊断的模型。雷达系统电路仿真结果表明了模拟电路的小波变换和最小二乘小波支持向量机故障诊断方法取得了较好的效果。     

Instability detection of ARMA systems based on AR system identification

The ability of parametric autoregressive (AR) system identification methods to detect the instability of an autoregressive moving average (ARMA) system of an unknown order is investigated. The collection of least squares AR estimators of various orders is shown to have the capacity to detect the instability of the underlying system. Necessary information is not the order of the system but, instead, an upper bound of the number of unstable poles with the maximal magnitude outside the unit circle.     

最小二乘双支持向量机的在线学习算法

针对具有两个非并行分类超平面的最小二乘双支持向量机,提出了一种在线学习算法。通过利用矩阵求逆分解引理,所提在线学习算法能充分利用历史的训练结果,避免了大型矩阵的求逆计算过程,从而降低了计算的复杂性。仿真结果验证了所提学习算法的有效性。     

Dual least squares support vector machines based spatiotemporal modeling for nonlinear distributed thermal processes

In this paper, a dual least squares support vector machines (LS-SVM) is proposed to model the thermal process. The infinite-dimensional system is first transformed into a finite-dimensional system through space-time separation. Then, the dual LS-SVM model is to approximate the two nonlinearities embedded in the system. Through space-time synthesis, the dual LS-SVM based spatiotemporal model is able to approximate the complex DPS with inherent coupled nonlinearities. The generalization performance of the proposed model is discussed using Rademacher complexity. Finally, simulations on a curing process demonstrate the effectiveness of the proposed modeling method.     

Nonlinear system identification with limited time data

With disturbances modeled by arbitrary solutions to a linear homogeneous differential equation, a least squares-equation error method is developed for parameter identification using data over a limited time interval which has application to certain classes of nonlinear and time varying systems. Examples include the Duffing, Hammerstein, Mathieu and Van der Pol equations together with a class of bilinear systems. The technique seeks to determine the parameters characterizing the disturbance modes in addition to the system parameters, based on the input-output data collected over the finite time interval. The approach circumvents the need to estimate unknown initial conditions through the use of a certain projection operator. Computational considerations are discussed and simulation results are summarized for the Van der Pol equation.     

An efficient hierarchical identification method for general dual-rate sampled-data systems

For the lifted input–output representation of general dual-rate sampled-data systems, this paper presents a decomposition based recursive least squares (D-LS) identification algorithm using the hierarchical identification principle. Compared with the recursive least squares (RLS) algorithm, the proposed D-LS algorithm does not require computing the covariance matrices with large sizes and matrix inverses in each recursion step, and thus has a higher computational efficiency than the RLS algorithm. The performance analysis of the D-LS algorithm indicates that the parameter estimates can converge to their true values. A simulation example is given to confirm the convergence results.