基于互信息的选择性集成核极端学习机

针对集成学习中的准确性和差异性平衡问题, 提出一种基于信息论的选择性集成核极端学习机. 采用具有结构简单、训练简便、泛化性能好的核极端学习作为基学习器. 引入相关性准则描述准确性, 冗余性准则描述差异性,将选择性集成问题转化为变量选择问题. 利用基于互信息的最大相关最小冗余准则对生成的核极端学习机进行选择, 从而实现准确性和差异性的平衡. 基于UCI 基准回归和分类数据的仿真结果验证了所提出算法的优越性.

     

基于递归最小二乘支持向量机的动态系统建模研究

针对递归最小二乘支持向量机的递归性易导致建模中偏微分方程组求解困难的问题,提出用解析法求解偏微分方程组,实现了完整的递归最小二乘支持向量机模型．首先分析了各参数的相关性,然后推导出偏微分方程的解析表达式并求解．仿真实例表明,在动态系统建模中,该模型的性能比常用的串并联模型以及现有不完整递归最小二乘支持向量机模型的精度更高、性能更好．

     

一种改进的在线最小二乘支持向量机回归算法

针对一般最小二乘支持向量机处理大规模数据集会出现训练速度慢,计算量大,不易在线训练的缺点,将修正后的遗忘因子矩形窗方法与支持向量机相结合,提出一种基于改进的遗忘因子矩形窗算法的在线最小二乘支持向量机回归算法,既突出了当前窗口数据的作用,又考虑了历史数据的影响.所提出的算法可减少计算量,提高在线辨识精度.仿真算例表明了该方法的有效性.

     

基于柯西分布加权的最小二乘支持向量机

针对Suykens等提出的加权最小二乘支持向量机(WLS-SVM)回归建模的不足和防止辨识模型的"过拟合",利用柯西分布函数的一些特性,提出了基于柯西分布加权的最小二乘支持向量机.根据预测误差的统计特性,以确定加权规则的参数,从而赋予训练样本不同的权值.由于考虑了生产过程中样本的实际特性,与已有的加权方法相比,新的加权最小二乘支持向量机更具有鲁棒性.仿真结果验证了该方法的可行性和有效性.

     

一种基于互信息变量选择的极端学习机算法

针对回归问题中存在的变量选择和网络结构设计问题, 提出一种基于互信息的极端学习机(ELM) 训练算法, 同时实现输入变量的选择和隐含层的结构优化. 该算法将互信息输入变量选择嵌入到ELM网络的学习过程之中, 以网络的学习性能作为衡量输入变量与输出变量相关与否的指标, 并以增量式的方法确定隐含层节点的规模.在Lorenz、Gas Furnace 和10 组标杆数据上的仿真结果表明了所提出算法的有效性. 该算法不仅可以简化网络结构, 还可以提高网络的泛化性能.

     

基于特征值分解的最大间隔支持向量回归机

广义特征值中心支持向量回归机(GEPSVR) 是一种有效的核回归算法, 但其在求解优化问题时易导致奇异 性问题. 为此, 提出一种基于特征值分解的支持向量回归机, 简称IGEPSVR. 与GEPSVR 相比, IGEPSVR 的主要优势 有: 结合最大间隔准则和GEPSVR 几何思想给出了新的距离度量准则; 在优化模型中引入Tikhonov 正则项, 克服了 可能产生的奇异性问题; IGEPSVR 仅需求解两个标准特征值, 降低了计算复杂度. 实验结果表明, 较GEPSVR 算法, IGEPSVR 不仅提高了学习能力, 而且缩短了训练时间.

     

一种基于D-S 和ARIMA 的多模型软测量方法

针对传统软测量方法存在的预测性能差、融合能力低等缺点, 提出一种基于证据理论(D-S) 合成规则和差分自回归滑动平均(ARIMA) 模型的多模型软测量方法. 首先利用自适应模糊核聚类方法和最小二乘支持向量机建立多个子模型; 然后利用D-S 合成规则构造的概率分配函数作为权值因子, 对子模型输出进行融合以得到多模型的输出; 最后结合ARIMA 模型对静态多模型输出进行动态校正. 仿真研究与工业应用的结果表明, 所提出的方法具有良好的预测性能和融合能力.

     

混合核函数稀疏LS-SVM软测量建模与应用

针对最小二乘支持向量机存在的稀疏性欠缺和单核函数局限性问题,本文提出一种基于混合核函数稀疏最小二乘支持向量机的软测量建模方法.该方法使用多项式核函数和RBF核函数线性加权构成混合核函数,兼顾最小二乘支持向量机的全局拟合能力与局部拟合能力,以矢量基学习作为稀疏解算法,改善最小二乘支持向量机的稀疏性,在精简模型结构的同时,避免冗余信息中的噪声过多的拟合到模型参数中,进而采用粒子群算法优化模型部分参数.将此方法分别应用于Mackey- Glasss混沌模型的时间序列预测和乙烯精馏塔塔釜乙烯浓度预测,应用结果表明该方法较最小二乘支持向量机、稀疏最小二乘支持向量机以及混合核最小二乘支持向量机具有更好的泛化效果和预报精度,兆示出其良好的应用潜力.     

多变量系统的耦合梯度辨识算法与性能分析

针对多变量系统维数大、参数多、一般的辨识算法计算量大的问题, 基于耦合辨识概念, 推导多变量系统的耦合随机梯度算法, 利用鞅收敛定理分析算法的收敛性能. 算法的主要思想是将系统模型分解为多个单输出子系统,在子系统的递推辨识过程中, 将每个子系统的参数估计值耦合起来. 所提出算法与最小二乘算法和耦合最小二乘算法相比, 具有较少的计算量, 收敛速度可以通过引入遗忘因子得到改善. 性能分析表明了所提出算法收敛, 仿真实例验证了算法的有效性.

     

一种基于Cholesky分解的动态无偏LS-SVM学习算法

针对最小二乘支持向量机用于在线建模时存在的计算复杂性问题,提出一种动态无偏最小二乘支持向量回归模型.该模型通过改进标准最小二乘支持向量机结构风险的形式消除了偏置项.得到了无偏的最小二乘支持向量机,简化了回归系数的求解.根据模型动态变化过程中核函数矩阵的特点,设计了基于Cholesky分解的在线学习算法.该算法能充分利用历史训练结果,减少计算复杂性.仿真实验表明了所提出模型的有效性.     

Online chaotic time series prediction using unbiased composite kernel machine via Cholesky factorization

The kernel method has proved to be an effective machine learning tool in many fields. Support vector machines with various kernel functions may have different performances, as the kernels belong to two different types, the local kernels and the global kernels. So the composite kernel, which can bring more stable results and good precision in classification and regression, is an inevitable choice. To reduce the computational complexity of the kernel machine’s online modeling, an unbiased least squares support vector regression model with composite kernel is proposed. The bias item of LSSVR is eliminated by improving the form of structure risk in this model, and then the calculating method of the regression coefficients is greatly simplified. Simultaneously, through introducing the composite kernel to the LSSVM, the model can easily adapt to the irregular variation of the chaotic time series. Considering the real-time performance, an online learning algorithm based on Cholesky factorization is designed according to the characteristic of extended kernel function matrix. Experimental results indicate that the unbiased composite kernel LSSVR is effective and suitable for online time series with both the steep variations and the smooth variations, as it can well track the dynamic character of the series with good prediction precisions, better generalization and stability. The algorithm can also save much computation time comparing to those methods using matrix inversion, although there is a little more loss in time than that with the usage of single kernels.     

一种多元核Logistic回归说话人辨别方法

针对文本无关话者辨别多分类目标和大训练样本情况,将经典Logistic回归模型进行多元化变形,并叠加L2惩罚因子以提高模型泛化能力.将最优目标负对数Logistic公式对偶化,并利用序列最小优化算法进行模型训练,速率优于传统多元核Logistic回归训练算法.实验显示,该模型构建简单,训练算法快捷,且识别率优于经典支持向量机与二元核Logistic回归模型所生成的"一对一"多分类方法.     

An unbiased LSSVM model for classification and regression

Aiming at the common support vector machine’s biased disadvantage and computational complexity, an unbiased least squares support vector machine (LSSVM) model is proposed in this paper. The model eliminates the bias item of LSSVM by improving the form of structure risk, then the unbiased least squares support vector classifier and the unbiased least squares support vector regression are deduced. Based on this model, we design a new learning algorithm using Cholesky factorization according to the characteristic of kernel function matrix, in this way the calculation of Lagrangian multipliers is greatly simplified. Several experiments on diffenert datasets are carried out, including the common datasets classification, synthetic aperture radar image automatic target recognition and chaotic time series prediction. The experimental results of correct recognition rate and the fitting precision testify that the unbiased LSSVM model has good universal ability and fitting accuracy, better generalization capability and stability, and have a great improvement in learning speed.     

A MKL based on-line prediction for gasholder level in steel industry

The real-time prediction for gasholder level is significant for gas scheduling in steel enterprises. In this study, we extended the least squares support vector regression (LSSVR) to multiple kernel learning (MKL) based on reduced gradient method. The MKL based LSSVR, using the optimal linear combination of kernels, improves the generalization of the model and reduces the training time. The experiments using the classical non-flat function and the practical problem shows that the proposed method achieves well performance and high computational efficiency. And, an application system based on the approach is developed and applied to the practice of Shanghai Baosteel Co. Ltd.     

利用矢量基学习和自适应迭代算法改进LSSVR

针对最小二乘支持向量回归缺乏传统SVR的稀疏性和鲁棒性等问题,综合矢量基学习和自适应迭代算法的优势,提出了一种改进的加权最小二乘支持向量回归算法（LSSVR）。该算法通过引入用矢量基学习和自适应迭代相结合的方式得到一个小的支持向量集,可以避免递推时可能出现的误差积累问题,有效提高算法的稀疏性和稳定性;同时采用加权方法确定权值系数以减小训练样本中非高斯噪声的影响。实验结果表明,改进的LSSVR具有较好的鲁棒性、支持向量稀疏性和动态建模实时性。     

回归最小二乘支持向量机的增量和在线式学习算法

首先给出回归最小二乘支持向量机的数学模型,并分析了它的性质,然后在此基础上根据分块矩阵计算公式和核函数矩阵本身的特点设计了支持向量机的增量式学习算法和在线学习算法．该算法能充分利用历史的训练结果,减少存储空间和计算时间．仿真实验表明了这两种学习方法的有效性．     

USING LEAST SQUARE SVM FOR NONLINEAR SYSTEMS MODELING AND CONTROL

Support vector machine is a learning technique based on the structural risk minimization principle, and it is also a class of regression method with good generalization ability. The paper firstly introduces the mathematical model of regression least squares support vector machine (LSSVM), and designs incremental learning algorithms by the calculation formula of block matrix, then uses LSSVM to model nonlinear system, based on which to control nonlinear systems by model predictive method. Simulation experiments indicate that the proposed method provides satisfactory performance, and it achieves superior modeling performance to the conventional method based on neural networks, moreover it achieves well control performance.     

基于混合最小二乘支持向量机网络模型的非线性系统辨识

针对基于输入输出数据的非线性系统辨识问题,提出一种新的混合最小二乘支持向量机(LS-SVMs)网络模型及相应的学习算法.该算法将系统的辨识问题动态自适应的划分为若干子问题,将支持向量机(SVM)用于各子模块辨识;通过分析模型的统计学特性,给出基于整体框架优化的系统参数辨识方法.针对系统中参数相关联的特性,采用期望条件最大化(ECM)算法对其进行条件辨识,同时结合正则化理论和最小二乘法,保证各专家模块的结构风险最小化辨识原则.试验结果表明,该方法兼具良好的辨识精度和泛化性能.     

Least squares support vector machines with tuning based on chaotic differential evolution approach applied to the identification of a thermal process

In the past decade, support vector machines (SVMs) have gained the attention of many researchers. SVMs are non-parametric supervised learning schemes that rely on statistical learning theory which enables learning machines to generalize well to unseen data. SVMs refer to kernel-based methods that have been introduced as a robust approach to classification and regression problems, lately has handled nonlinear identification problems, the so called support vector regression. In SVMs designs for nonlinear identification, a nonlinear model is represented by an expansion in terms of nonlinear mappings of the model input. The nonlinear mappings define a feature space, which may have infinite dimension. In this context, a relevant identification approach is the least squares support vector machines (LS-SVMs). Compared to the other identification method, LS-SVMs possess prominent advantages: its generalization performance (i.e. error rates on test sets) either matches or is significantly better than that of the competing methods, and more importantly, the performance does not depend on the dimensionality of the input data. Consider a constrained optimization problem of quadratic programing with a regularized cost function, the training process of LS-SVM involves the selection of kernel parameters and the regularization parameter of the objective function. A good choice of these parameters is crucial for the performance of the estimator. In this paper, the LS-SVMs design proposed is the combination of LS-SVM and a new chaotic differential evolution optimization approach based on Ikeda map (CDEK). The CDEK is adopted in tuning of regularization parameter and the radial basis function bandwith. Simulations using LS-SVMs on NARX (Nonlinear AutoRegressive with eXogenous inputs) for the identification of a thermal process show the effectiveness and practicality of the proposed CDEK algorithm when compared with the classical DE approach.     

基于带特征染色体遗传算法的支持向量机特征选择和参数优化

鉴于支持向量机特征选择和参数优化对其分类准确率有重大的影响,将支持向量机渐近性能融入遗传算法并生成特征染色体,从而将遗传算法的搜索导向超参数空间中的最佳化误差直线.在此基础上,提出一种新的基十带特征染色体遗传算法的方法,同时进行支持向量机特征选择和参数优化.在与网格搜索、不带特征染色体遗传算法和其他方法的比较中,所提出的方法具有较高的准确率、更小的特征子集和更少的处理时间.