在线鲁棒最小二乘支持向量机回归建模

鉴于工业过程的时变特性以及现场采集的数据通常具有非线性特性且包含离群点,利用最小二乘支持向量机回归（least squares support vector regression,LSSVR）建模易受离群点的影响.针对这一问题,结合鲁棒学习算法（robust learning algorithm,RLA）,本文提出了一种在线鲁棒最小二乘支持向量机回归建模方法.该方法首先利用LSSVR模型对过程输出进行预测,与真实输出相比较得到预测误差;然后利用RLA方法训练LSSVR模型的权值,建立鲁棒LSSVR模型;最后应用增量学习方法在线更新鲁棒LSSVR模型,从而得到在线鲁棒LSSVR模型.仿真研究验证了所提方法的有效性.     

鲁棒的加权核主成分分析算法

为减弱离群点对数据处理的影响, 提出了一种鲁棒的加权核主成分分析算法。利用核函数将样本投影到核空间, 在核空间构建一个样本加权重建误差最小模型, 最大限度地提取数据中的非线性信息并降低离群点样本的干扰。在Yale人脸库和UCI数据集上的实验表明, 该方法具有很好的识别率, 尤其对离群点样本具有较好的鲁棒性。     

基于投影寻踪的非线性鲁棒偏最小二乘法及应用

来自工业现场的数据往往具有非线性特性且包含离群点, 利用非线性偏最小二乘(partial least squares, PLS)建模易受离群点的影响. 针对这一问题, 结合径向基函数(radial basis function, RBF)网络, 本文提出了一种基于投影寻踪的非线性鲁棒PLS方法. 该方法首先利用RBF变换将自变量与因变间的非线性关系转化为线性关系; 然后利用投影寻踪算法提取变换后自变量的鲁棒偏最小二乘法成分; 最后建立鲁棒PLS成分与因变量之间的鲁棒线性回归模型. 将该方法应用于湿法冶金萃余液pH值软测量建模问题, 结果验证了其有效性.     

基于鲁棒加权模糊聚类的污水处理过程监测方法

针对非线性强、先验故障知识少、异常工况识别难的污水处理过程监测问题,提出一种基于鲁棒加权模糊c均值(Robust weighted fuzzy c-means, RoW-FCM)聚类与核偏最小二乘(Kernel partial least squares, KPLS)的过程监测方法.首先,针对污水处理过程的高维非线性耦合特性,采用核偏最小二乘对高维输入变量进行降维;其次,针对传统基于最近邻分配的模糊c均值算法对离群点敏感以及存在聚类不平衡簇的问题,提出充分考虑样本间相互关系的基于鲁棒加权模糊c均值聚类算法.通过引入可能性划分矩阵作为权值参数实现不同样本数据的区分加权,提高了离群点数据聚类的鲁棒性,同时引入聚类大小控制参数解决不平衡簇的问题.进一步将基于鲁棒加权模糊c均值算法对核偏最小二乘降维后的得分矩阵进行聚类,利用聚类得到的隶属度矩阵实现异常工况的检测;最后,建立隶属度矩阵与过程变量的回归模型,并利用得到的变量贡献矩阵描述变量对各个簇的解释程度,实现异常工况的识别.数值仿真以及污水处理过程数据实验表明该方法具有更好的鲁棒性能,在异常工况检测和识别上具有较好的效果.     

鲁棒PLS在间歇生产过程监控中的应用

间歇和半间歇过程在化学工业中占有重要地位,如何对其进行监控一直是过程控制领域研究的热点之一.现实过程中,数据大都存在离群点,易使多向部分最小二乘(MPLS)模型造成误差.针对MPLS统计监控受离群点影响的问题,提出一种基于鲁棒MPLS的统计监控分析和相应鲁棒监控统计量的计算方法.相对于普通MPLS,鲁棒MPLS在建模数据中存在离群点时仍能给出正确的统计监控模型,降低了建模过程对数据的要求.     

高炉铁水质量鲁棒正则化随机权神经网络建模

高炉炼铁过程运行优化与控制依赖于可靠、稳定的难测铁水质量(Molten iron quality, MIQ)指标模型.针对现有MIQ建模方法的不足, 本文提出一种新型的数据驱动鲁棒正则化随机权神经网络(Random vector functional-link networks, RVFLNs)算法, 用于实现MIQ指标在线估计的鲁棒建模.首先, 为了提高建模效率和降低计算复杂度, 采用数据驱动典型相关性分析方法从众多变量中提取与MIQ相关性最强的变量作为建模输入变量; 其次, 由于传统RVFLNs网络的输出权值由最小二乘估计获得, 易受离群数据影响而鲁棒性差, 引入基于Gaussian分布加权的M估计技术, 提出新型鲁棒RVFLNs算法建立多元MIQ指标的鲁棒模型; 同时, 在鲁棒加权后的最小二乘损失函数基础上, 进一步引入${L_1}$和${L_2}$两个正则化项以构成优化目标函数的Elastic net, 用于稀疏化RVFLNs网络的输出权值矩阵, 解决RVFLNs网络多重共线性和过拟合的问题.最后, 基于某大型高炉工业数据, 进行充分数据实验, 结果表明所提方法具有更高的建模与估计精度以及较强的鲁棒性能.     

一种基于鲁棒估计的极限学习机方法

极限学习机(ELM)是一种单隐层前馈神经网络(single-hidden layer feedforward neural networks,SLFNs),它相较于传统神经网络算法来说结构简单,具有较快的学习速度和良好的泛化性能等优点。ELM的输出权值是由最小二乘法(least square,LE)计算得出,然而经典的LS估计的抗差能力较差,容易夸大离群点和噪声的影响,从而造成训练出的参数模型不准确甚至得到完全错误的结果。为了解决此问题,提出一种基于M估计的采用加权最小二乘方法来取代最小二乘法计算输出权值的鲁棒极限学习机算法(RBELM),通过对多个数据集进行回归和分类分析实验,结果表明,该方法能够有效降低异常值的影响,具有良好的抗差能力。     

基于自适应加权最小二乘支持向量机的芳烃产量软测量建模

芳烃收率是催化重整生产过程中的重要质量指标。针对其软测量建模中样本数据可能存在的测量误差对模型性能的影响,提出一种自适应加权最小二乘支持向量机(AWLSSVM)回归建模方法。该方法基于最小二乘支持向量机模型,根据样本拟合误差,并结合改进的指数分布加权规则,为每个建模样本分配不同的权值,以降低测量误差对建模精度的影响;同时提出一种全局优化算法—混沌粒子群模拟退火(CPSO-SA)算法对最小二乘支持向量机的模型参数进行优化选择,以提高模型的泛化能力。仿真实验表明,AWLS-SVM模型的预测精度及鲁棒性能优于LS-SVM和WLS-SVM。最后,应用AWLS-SVM方法建立催化重整生产过程芳烃收率的软测量模型,获得了较好的效果。     

基于前向查找和均值漂移的点模型鲁棒降噪算法

针对点模型提出了基于前向查找和均值漂移两种鲁棒统计方法的滤波算法。前向查找算法根据残差图自动检测离群点,并将输入的点云数据划分为多个不带离群点的最优局部降噪邻域。对局部邻域进行加权协方差分析,估计出该邻域的最小二乘拟合平面。在局部邻域内估计采样点的核密度函数并通过均值漂移算法计算它的局部最大值点,核密度函数的局部最大值点确定了点云数据的聚类中心并能准确逼近采样点曲面,将每一个采样点漂移到密度函数的局部最大值点,使点云曲面收敛为一个稳定的三维数字模型。实验结果表明,本文的算法是鲁棒的,能在有效剔除点模型表面噪声的同时较好地保持模型表面的尖锐特征。     

基于准最小二乘法的光伏阵列模型鲁棒参数估计方法

光伏阵列的模型参数估计在光伏发电系统的仿真、输出功率预测、最大功率点跟踪等方面有重要意义。当测量数据中只含随机误差时,以加权最小二乘（WLS）为优化函数的参数估计方法有较好的辩识效果。但是当测量数据中含有显著误差时,WLS参数辩识的效果较差。为解决此问题,本文提出了一种以准加权最小二乘法（QWLS）为优化函数的参数估计方法来减小显著误差的影响,采用了赤池信息量准则(AIC)设计QWLS最优参数,将该方法应用于光伏阵列中构造模型鲁棒参数估计问题。最后将WLS和QWLS分别结合序列二次规划（SQP）算法,进行光伏阵列模型的参数估计仿真与实验测试。测试结果显示QWLS参数估计结果更准确,验证了准最小二乘法的鲁棒性与有效性。     

Robust least squares support vector machine based on recursive outlier elimination

To achieve robust estimation for noisy data set, a recursive outlier elimination-based least squares support vector machine (ROELS-SVM) algorithm is proposed in this paper. In this algorithm, statistical information from the error variables of least squares support vector machine is recursively learned and a criterion derived from robust linear regression is employed for outlier elimination. Besides, decremental learning technique is implemented in the recursive training–eliminating stage, which ensures that the outliers are eliminated with low computational cost. The proposed algorithm is compared with re-weighted least squares support vector machine on multiple data sets and the results demonstrate the remarkably robust performance of the ROELS-SVM.     

Weighted and robust archetypal analysis

Archetypal analysis represents observations in a multivariate data set as convex combinations of a few extremal points lying on the boundary of the convex hull. Data points which vary from the majority have great influence on the solution; in fact one outlier can break down the archetype solution. The original algorithm is adapted to be a robust M-estimator and an iteratively reweighted least squares fitting algorithm is presented. As a required first step, the weighted archetypal problem is formulated and solved. The algorithm is demonstrated using an artificial example, a real world example and a detailed simulation study.     

基于最小p-范数的宽度学习系统

在宽度学习系统的基础上,以误差矢量的p-范数为损失函数,结合固定点迭代策略,提出基于最小p-范数的宽度学习系统.通过灵活设置p的取值(p≥1),提出的最小p-范数宽度学习系统能较好应对不同噪声的干扰,实现对不确定数据的建模任务.数值实验表明,在高斯、均匀、脉冲噪声干扰环境下,文中系统均能保持良好性能.将该系统应用于脑电图分类任务,在大多数被试上都能取得较高的分类精度.     

基于局部敏感判别宽度学习的高光谱图像分类

宽度学习系统(BLS)以其良好的学习性能与泛化能力,在高光谱图像(HSI)分类中得到了广泛应用。然而宽度学习系统仅关注各类样本的可分性,忽略了样本之间的相对关系以及所蕴涵的判别信息,在一定程度上限制了宽度学习系统在高光谱图像分类任务中的性能。为此,提出一种局部敏感判别的宽度学习系统(LSDBLS)方法。该方法通过引入局部敏感判别分析考虑标记样本的判别信息与数据样本的局部流形结构,通过标记样本构建类内图和类间图来表征数据样本之间的相对关系。在此基础上,将类内图和类间图引入到宽度学习系统的目标函数中,通过最小化类内图以及最大化类间图,使得同类样本尽可能地聚集,不同类的样本尽可能地远离,增强LSDBLS对数据特征的判别能力。通过在三个HSI数据集上的实验结果表明,LSDBLS取得了良好的效果。     

基于深度学习的仿生集群运动智能控制

宽度学习系统(BLS)是一种基于RVFLN的高效增量学习系统,具有快速且精度高的特点.为了实现BLS对时间序列的精确预测,结合回声状态网络(ESN)的储备池结构,提出一种基于池计算的宽度学习系统(RCBLS).该系统通过在强化层引入简单环型储备池连接,以并行的储备池代替原系统中的前馈连接,使RCBLS具有一定的回声状态特性且方便设计.同时,应用增量学习保证了系统的实时性能.基于MSO时间序列预测问题,针对不同规模数据样本分别研究不同储备池结构RCBLS的性能.结果表明:多储备池结构的RCBLS大大提高了模型的泛化能力和稳定性.     

M-estimator and D-optimality model construction using orthogonal forward regression.

This correspondence introduces a new orthogonal forward regression (OFR) model identification algorithm using D-optimality for model structure selection and is based on an M-estimators of parameter estimates. M-estimator is a classical robust parameter estimation technique to tackle bad data conditions such as outliers. Computationally, The M-estimator can be derived using an iterative reweighted least squares (IRLS) algorithm. D-optimality is a model structure robustness criterion in experimental design to tackle ill-conditioning in model structure. The orthogonal forward regression (OFR), often based on the modified Gram-Schmidt procedure, is an efficient method incorporating structure selection and parameter estimation simultaneously. The basic idea of the proposed approach is to incorporate an IRLS inner loop into the modified Gram-Schmidt procedure. In this manner, the OFR algorithm for parsimonious model structure determination is extended to bad data conditions with improved performance via the derivation of parameter M-estimators with inherent robustness to outliers. Numerical examples are included to demonstrate the effectiveness of the proposed algorithm.     

Locally and globally robust Penalized Trimmed Squares regression

Multiple outliers are frequently encountered in regression models used in business, economics, engineers and applied studies. The ordinary least squares (OLS) estimator fails even in the presence of a single outlying observation. To overcome this problem, a class of high breakdown robust estimators (insensitive to outliers up to 50% of the data sample) has been introduced as an alternative to the least squares regression. Among them the Penalized Trimmed Squares (PTS) is a reasonable high breakdown estimator. This estimator is defined by the minimization of an objective function where penalty cost for deleting an outlier is added, which serves as an upper bound on the residual error for any feasible regression line. Since the PTS does not require presetting the number of outliers to delete from the data set, it has better efficiency with respect to other estimators. However, small outliers remain influential causing bias to the regression line. In this work we present a new class of regression estimates called generalized PTS (GPTS). The new GPTS estimator is defined as the PTS but with penalties suitable for bounding the influence function of all observations. We show with some numerical examples and a Monte Carlo simulation study that the generalized PTS estimate has very good performance for both robust and efficiency properties.     

A rapid supervised learning neural network for functioninterpolation and approximation

This paper presents a neural-network architecture and an instant learning algorithm that rapidly decides the weights of the designed single-hidden layer neural network. For an n-dimensional N-pattern training set, with a constant bias, a maximum of N-r-1 hidden nodes is required to learn the mapping within a given precision (where r is the rank, usually the dimension, of the input patterns). For off-line training, the proposed network and algorithm is able to achieve "one-shot" training as opposed to most iterative training algorithms in the literature. An online training algorithm is also presented. Similar to most of the backpropagation type of learning algorithms, the given algorithm also interpolates the training data. To eliminate outlier data which may appear in some erroneous training data, a robust weighted least squares method is proposed. The robust weighted least squares learning algorithm can eliminate outlier samples and the algorithm approximates the training data rather than interpolates them. The advantage of the designed network architecture is also mathematically proved. Several experiments show very promising results.     

A novel decomposition-based ensemble model for short-term load forecasting using hybrid artificial neural networks

Highly accurate short-term load forecasting (STLF) is essential in the operation of power systems. However, the existing predictive methods cannot achieve an effective balance between prediction accuracy and computational cost. Furthermore, the prediction residual is rarely used to improve the predictive accuracy in STLF. This paper proposes a novel decomposition-based ensemble model for the STLF task. First, an optimized empirical wavelet transform (OEWT) is developed to rationally decompose the STLF load by combining the approximate entropy method with the empirical wavelet transform. Particularly, OEWT improves both prediction accuracy and computational cost in STLF. Second, a new hybrid machine learning method (named master learner) is proposed by rationally combining long short-term memory networks (LSTMs) with broad learning system (BLS) in STLF, effectively strengthening the predictive accuracy without significantly increasing the computational cost. Third, a residual learning model (named residual learner) is developed in the master learner to extract the effective predictive information from residual results, further improving the prediction accuracy in STLF. Fourth, an auxiliary learner is proposed by introducing another BLS to connect the input and output of the proposed model, enhancing the predictive robustness. The proposed decomposition-based ensemble model is compared with state-of-the-art and traditional models in STLF. Experimental results show that the model not only has high predictive accuracy and robustness but also low computational cost.

     

M-estimator with asymmetric influence function for estimating the Burr type III parameters with outliers

The Burr type III distribution allows for a wider region for the skewness and kurtosis plane, which covers several distributions including the log-logistic, and the Weibull and Burr type XII distributions. However, outliers may occur in the data set. The robust regression method such as an M-estimator with symmetric influence function has been successfully used to diminish the effect of outliers on statistical inference. However, when the data distribution is asymmetric, these methods yield biased estimators. We present an M-estimator with asymmetric influence function (AM-estimator) based on the quantile function of the Burr type III distribution to estimate the parameters for complete data with outliers. The simulation results show that the M-estimator with asymmetric influence function generally outperforms the maximum likelihood and traditional M-estimator methods in terms of the bias and root mean square errors. One real example is used to demonstrate the performance of our proposed method.