基于优化核极限学习机的中期电力负荷预测

针对电力负荷预测,提出了一种优化的核极限学习机(O-KELM)的方法.核极限学习机(KELM)方法仅以核函数表示未知的隐含层非线性特征映射,无需选择隐含层的节点数目,通过正则化最小二乘算法计算网络的输出权值.将优化算法应用于KELM方法中,给出基于遗传算法、微分演化、模拟退火的3种优化KELM方法,优化选择核函数的参数以及正则化系数,以进一步提高KELM方法的学习性能.为验证方法的有效性,将O-KELM方法应用于某地区的中期峰值电力负荷预测研究中,在同等条件下与优化极限学习机(O-ELM)方法、SVM等方法进行比较.实验结果表明,O-KELM方法具有很好的预测性能,其中GA-KELM方法的建模精度最高.     

全相关核偏最小二乘故障诊断方法及在抽油机上应用

针对油田抽油机生产数据存在强非线性和强耦合性, 导致故障诊断困难的问题, 本文提出一种全相关动态 核偏最小二乘(FCDKPLS)故障诊断方法. 首先, 构建抽油机生产数据自回归模型, 反映数据变量间的动态特性; 其 次, 分析了KPLS算法中输出变量与输入变量残差子空间的相关性, 为此, 在输出模型上构建一个辅助矩阵, 从而表 征输入变量与输出变量的全相关性, 建立输入变量和输出变量之间更直接的联系. 最后, 将提出的全相关动态偏最 小二乘方法应用于抽油机过程故障诊断, 实验结果表明本文提出方法的有效性.     

基于SOA-LSSVM的短时交通流量预测

针对短时交通流量存在的非线性与不确定性的问题,结合搜索者算法收敛精度高和最小二乘支持向量机计算速度快的优点,提出基于搜索者-最小二乘支持向量机( SOA-LSSVM)的流量预测模型,将该模型应用于短时交通流量预测,并与人工神经网络进行对比分析,结果表明,该模型具有较高的预测精度和泛化能力,适合于短时交通流量的预测,具有良好的推广应用价值。     

基于KRLS的pH中和过程建模

针对典型的pH酸碱中和过程,提出基于核递推最小二乘(KRLS)的核学习动态模型。KRLS方法采用基于近似线性依赖技术的稀疏化算法,降低了计算复杂度及存储量,能适用于较大规模数据集的训练以及动态时变过程的建模。将所提方法应用到具有缓冲流的双输出中和过程实例中,为验证其有效性,在同等条件下,还与核偏最小二乘(KPLS)、核主成分分析—支持向量机(KPCA-SVM)、核极限学习机(KELM)、极限学习机(ELM)、支持向量机(SVM)等方法进行比较。实验结果表明:作为一种在线自适应方法,KRLS方法具有很高的动态建模精度,为研究pH中和过程的控制奠定了基础。     

基于KPLS-LSSVM的航煤闪点和干点软测量应用研究

软测量技术是解决流程工业中一类难以在线测量变量估计问题的有效方法。本文提出了一种基于核偏最小二乘方法(KPLS)和最小二乘支持向量机(LS-SVM)的航煤闪点和干点软测量建模方法,采用核偏最小二乘方法对输入变量进行数据压缩,消除变量之间的相关性,简化最小二乘支持向量机模型结构,并通过混沌粒子群—模拟退火方法(CPSO-SA)对最小二乘支持向量机的参数进行优化选择。将优化结果应用于CDU航煤闪点和干点软测量建模,结果表明:该方法具有学习速度快、跟踪性能好以及泛化能力强等优点,为分馏过程在线质量控制的实施奠定了基础。     

基于KPLS的网络入侵特征抽取及检测方法

从特征抽取的角度研究提高入侵检测性能问题,提出应用核偏最小二乘(KPLS)进行入侵特征抽取和检测的方法.其优点在于KPLS能非线性地抽取输入特征的多个正交分量,并保持与输出类别的相关性,可同时完成入侵特征抽取和判别.将该方法应用于基于Linux主机的入侵检测实验,取得了比SVM和KPCR等方法更好的效果.     

基于最小二乘法的SIMO傅里叶神经网络研究

利用傅里叶级数的原理，构造单输入、多输出(SIMO)傅里叶神经网络，将非线性映射转化成为线性映射，将求解神经网络权值的方法由非线性优化方法转化成为线性优化方法，并采用最小二乘法计算网络的权值，从而大大提高了神经网络的收敛速度并避免了局部极小问题.而且，在训练输出样本受白噪声影响时，最小二乘法具有良好的降低噪声影响的功能．     

基于混合KPLS-FDA的过程监控和质量预报方法

提出一种基于核偏最小二乘(KPLS)与费舍尔判别分析(FDA)相结合的过程监控和质量预报方法—–混合 KPLS-FDA 方法.首先,利用 KPLS 提取过程数据的非线性特征,使用 FDA 建立 KPLS 的内部模型;然后,求出满足最大分离度的核 Fisher 特征向量和判别向量来实现状态监测,若系统运行正常,则根据 KPLS 回归模型预报产品的质量,否则利用 Fisher 相似度系数确定故障类型;最后,通过轧钢过程的仿真研究验证了混合 KPLS-FDA 方法的有效性.     

基于KRLS的非均匀采样非线性系统辨识

利用提升技术可将非均匀采样非线性系统离散化为一个多输入单输出传递函数模型,从而将系统输出表示为非均匀刷新非线性输入和输出回归项的线性参数模型,进一步基于非线性输入的估计或过参数化方法进行辨识.然而,当非线性环节结构未知或不能被可测非均匀输入参数化表示时,上述辨识方法将不再适用.为了解决这个问题,利用核方法将原始非线性数据投影到高维特征空间中使其线性可分,再对投影后的数据应用递推最小二乘算法进行辨识,提出基于核递推最小二乘的非均匀采样非线性系统辨识方法.此外,针对系统含有有色噪声干扰的情况,参考递推增广最小二乘算法的思想,利用估计残差代替不可测噪声,提出核递推增广最小二乘算法.最后,通过仿真例子验证所提算法的有效性.     

基于全相关动态核偏最小二乘法的故障诊断方法

传统KPLS在构造输入变量与输出变量时未考虑样本间的动态特性,使故障样本信息易被其他样本掩盖;同时,KPLS未考虑所有变量对故障样本的影响,使变量间隐藏信息不能被完全表达。针对以上问题,提出一种基于全相关动态KPLS（all correlated dynamic KPLS,ADKPLS）的非线性工业过程故障诊断方法。该方法首先对原始数据进行动态特征分析研究,使组成的数据矩阵能很好地反映变量间的动态关系。然后分析证明了KPLS中输出变量的变化会影响到输入残差空间,设计一个输出变量辅助矩阵,表征输入变量与输出变量的全相关性。最后,采用基于输入变量与输出变量之间的全相关信息构建贡献图以识别故障源变量。与KPLS相比,在输入和输出变量之间建立了更直接的关系。在抽油机生产过程上的仿真结果表明,所提方法具有良好的故障检测与识别性能。     

A sparse kernel algorithm for online time series data prediction

Kernel based methods have been widely applied for signal analysis and processing. In this paper, we propose a sparse kernel based algorithm for online time series prediction. In classical kernel methods, the kernel function number is very large which makes them of a high computational cost and only applicable for off-line or batch learning. In online learning settings, the learning system is updated when each training sample is obtained and it requires a higher computational speed. To make the kernel methods suitable for online learning, we propose a sparsification method based on the Hessian matrix of the system loss function to continuously examine the significance of the new training sample in order to select a sparse dictionary (support vector set). The Hessian matrix is equivalent to the correlation matrix of sample inputs in the kernel weight updating using the recursive least square (RLS) algorithm. This makes the algorithm able to be easily implemented with an affordable computational cost for real-time applications. Experimental results show the ability of the proposed algorithm for both real-world and artificial time series data forecasting and prediction.     

A wavelet extreme learning machine

Extreme learning machine (ELM) has been widely used in various fields to overcome the problem of low training speed of the conventional neural network. Kernel extreme learning machine (KELM) introduces the kernel method to ELM model, which is applicable in Stat ML. However, if the number of samples in Stat ML is too small, perhaps the unbalanced samples cannot reflect the statistical characteristics of the input data, so that the learning ability of Stat ML will be influenced. At the same time, the mix kernel functions used in KELM are conventional functions. Therefore, the selection of kernel function can still be optimized. Based on the problems above, we introduce the weighted method to KELM to deal with the unbalanced samples. Wavelet kernel functions have been widely used in support vector machine and obtain a good classification performance. Therefore, to realize a combination of wavelet analysis and KELM, we introduce wavelet kernel functions to KELM model, which has a mix kernel function of wavelet kernel and sigmoid kernel, and introduce the weighted method to KELM model to balance the sample distribution, and then we propose the weighted wavelet–mix kernel extreme learning machine. The experimental results show that this method can effectively improve the classification ability with better generalization. At the same time, the wavelet kernel functions perform very well compared with the conventional kernel functions in KELM model.     

基于支持向量机的大样本回归算法比较研究

支持向量机的研究是当前人工智能领域的研究热点。基于支持向量机的大样本回归问题一直是一个非常具有挑战性的课题。最近,基于递归最小二乘算法,Engel等人提出了核递归最小二乘算法。文中基于块增量学习和逆学习过程,提出了自适应迭代回归算法。为了说明两种方法的性能,论文在训练速度、精度和支持向量数量等方面,对它们做了比较。模拟结果表明:核递归最小二乘算法所得到的支持向量个数比自适应迭代回归算法少,而训练时间比自适应迭代回归算法的训练时间长,训练和测试精度也比自适应迭代回归算法差。     

基于Xgboost优化的KELM滑坡预报模型研究

针对极限学习机对滑坡预测准确性低及在训练过程中模型不稳定的问题,引入RBF高斯核函数并使用极限梯度提升树算法Xgboost对KELM进行优化,建立了Xgboost优化后的Xgboost-KELM预测模型;首先采用高斯核RBF作为极限学习机的核函数,解决隐藏节点随机映射问题,增加模型稳定性及适用性;其次将清洗后的监测数据作为模型输入,并使用Xgboost寻优算法对核函数中的超参数进行优化,通过4组测试集进行Xgboost-KELM建模,依据均方误差迭代曲线得出最佳超参数;最后使用两组10%样本集验证模型评价指标及稳定性,实验结果AUC均值对比模型至少提高3个百分点,Precision、Accuracy及Recall至少高于对比模型1.7个百分点,同时Xgboost-KELM模型的方差及偏差都较小,证明该模型稳定性较好,实验结果说明Xgboost-KELM模型具有较好的预测效果,在滑坡灾害预测中有较好的预测能力。     

Bridging deep and multiple kernel learning: A review

Kernel methods and deep learning are two of the most currently remarkable machine learning techniques that have achieved great success in many applications. Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the reproducing kernel Hilbert space (RKHS) while making the computation tractable by leveraging the kernel trick. It is commonly agreed that the success of kernel methods is very much dependent on the choice of kernel. Multiple kernel learning (MKL) is one possible scheme that performs kernel combination and selection for a variety of learning tasks, such as classification, clustering, and dimensionality reduction. Deep learning models project input data through several layers of nonlinearity and learn different levels of abstraction. The composition of multiple layers of nonlinear functions can approximate a rich set of naturally occurring input-output dependencies. To bridge kernel methods and deep learning, deep kernel learning has been proven to be an effective method to learn complex feature representations by combining the nonparametric flexibility of kernel methods with the structural properties of deep learning. This article presents a comprehensive overview of the state-of-the-art approaches that bridge the MKL and deep learning techniques. Specifically, we systematically review the typical hybrid models, training techniques, and their theoretical and practical benefits, followed by remaining challenges and future directions. We hope that our perspectives and discussions serve as valuable references for new practitioners and theoreticians seeking to innovate in the applications of the approaches incorporating the advantages of both paradigms and exploring new synergies.     

短期风速多步预测的研究

为了提高风电场短期风速预测的精确度以及预测尺度,提出了一种将小波分解法、经验模式分解法及最小二乘支持向量机相结合对风速时间序列进行短期多步预测建模的方法。该方法采用小波分解法对风速信号进行分解,使之分解成不同频带的高频和低频分量;再利用最小二乘支持向量机对各分量建立预测模型,将各预测模型的预测值叠加可得到模型的预测结果。该模型称为预测模型Ⅰ。其次,将预测模型I的预测结果设为训练样本,采用经验模式分解法把训练样本集分解成若干本征模式分量和趋势项;再利用最小二乘支持向量机对各本征模式分量和趋势项建立预测模型,同时扩大模型的预测尺度;将各预测模型的预测值叠加可得该模型的预测结果。该模型称为预测模型Ⅱ。最后,将预测模型Ⅱ、Ⅰ的预测值叠加得到最终预测结果。实验结果表明,采用该方法预测的风电场短期风速的RMSE值为0.153,验证了该方法的有效性。     

基于CEEMDAN-FE-KELM方法的短期风电功率预测

针对短期风电功率预测,提出一种基于自适应噪声完整集成经验模态分解（complete ensemble empirical mode decomposition with adaptive noise,CEEMDAN）-模糊熵（FE）的核极限学习机（extreme learning machine with kernels,KELM）组合预测方法．CEEMDAN方法在信号分解的每一阶段都添加特定的白噪声,通过计算唯一的余量信号以获取各个模态分量,与EEMD（ensemble empirical mode decomposition）方法相比,其分解过程是完整的．为降低信号非平稳性对预测精度的影响及减少计算规模,采用CEEMDAN-模糊熵（FE）方法将信号分解为具有不同复杂度差异的子序列,然后分别构建相应的KELM预测模型,最后对预测结果进行合成．将CEEMDAN-FE-KELM方法应用于某地区的短期风电功率预测,在同等条件下,与单一的KELM方法及KELM的组合预测方法进行实验对比,结果证明该方法更有效．     

Sparse data-dependent kernel principal component analysis based on least squares support vector machine for feature extraction and recognition

Kernel learning is widely used in many areas, and many methods are developed. As a famous kernel learning method, kernel principal component analysis (KPCA) endures two problems in the practical applications. One is that all training samples need to be stored for the computing the kernel matrix during kernel learning. Second is that the kernel and its parameter have the heavy influence on the performance of kernel learning. In order to solve the above problem, we present a novel kernel learning namely sparse data-dependent kernel principal component analysis through reducing the training samples with sparse learning-based least squares support vector machine and adaptive self-optimizing kernel structure according to the input training samples. Experimental results on UCI datasets, ORL and YALE face databases, and Wisconsin Breast Cancer database show that it is feasible to improve KPCA on saving consuming space and optimizing kernel structure.     

基于KPCA和投影字典对学习的人脸识别算法

相比基于稀疏约束的字典学习算法和识别方法，投影字典对学习（projective Dictionary Pair Learning，DPL）具有更快的学习速度和更高的识别率.为了进一步提高DPL的识别能力，本文提出了改进DPL算法K-DPL，即将核主成分分析KPCA与DPL相结合的识别方法.在K-DPL算法中，利用核方法，将样本映射到高维空间以解决非线性问题，再进行DPL训练，得到更具判别性的字典.ORL库上实验表明，不同训练比下K-DPL相比DPL识别率至少提高了1.5%且识别速度提高了约20倍.在扩展YaleB和AR库上，K-DPL相比DPL识别率分别提高0.3%和0.4%，且识别速度有所提高，表明K-DPL对光照和遮挡具有较好的鲁棒性.