最小平方误差算法的正则化核形式

最小平方误差算法是最常用的一种经典模式识别和回归分析方法,其目标是使线性函 数输出与期望输出的误差平方和为最小.该文应用满足Meteer条件的核函数和正则化技术,改 造经典的最小平方误差算法,提出了基于核函数和正则化技术的非线性最小平方误差算法,即 最小平方误差算法的正则化核形式,其目标函数包含基于核的非线性函数的输出与期望输出的 误差平方和,及一个适当的正则项.正则化技术可以处理病态问题,同时可以减小解空间和控制 解的推广性,文中采用了三种平方型的正则项,并且根据正则项的概率解释,详细比较了三种正 则项之间的差别.最后,用仿真资料和实际资料进一步分析算法的性能.     

一种基于核函数的非线性感知器算法

为了提高经典Rosenblatt感知器算法的分类能力，该文提出一种基于核函数的非线性感知器算法，简称核感知器算法，其特点是用简单的迭代过程和核函数来实现非线性分类器的一种设计，核感知器算法能够处理原始属性空间中线性不可分问题和高维特征空间中线性可分问题。同时，文中详细分析了其算法与径向基函数神经网络、势函数方法和支持向量机等非线性算法的关系。人工和实际数据的计算结果表明：与线性感知器算法相比，核感知器算法可以有效地提高分类精度。     

低秩约束的非线性属性选择算法

针对高维的数据中往往存在非线性、低秩形式和属性冗余等问题,提出一种基于核函数的属性自表达无监督属性选择算法——低秩约束的非线性属性选择算法（LRNFS）。首先,将每一维的属性映射到高维的核空间上,通过核空间上的线性属性选择去实现低维空间上的非线性属性选择;然后,对自表达形式引入偏差项并对系数矩阵进行低秩与稀疏处理;最后,引入核矩阵的系数向量的稀疏正则化因子来实现属性选择。所提算法中用核矩阵来体现其非线性关系,低秩考虑数据的全局信息进行子空间学习,自表达形式确定属性的重要程度。实验结果表明,相比于基于重新调整的线性平方回归（RLSR）半监督特征选择算法,所提算法进行属性选择之后作分类的准确率提升了2.34%。所提算法解决了数据在低维特征空间上线性不可分的问题,提升了属性选择的准确率。     

自适应正则化核二维判别分析

传统的半监督降维技术中,在原特征空间中定义流形正则化项,但其构造无助于接下来的分类任务.针对此问题,文中提出一种自适应正则化核二维判别分析算法.首先每个图像矩阵经奇异值分解为两个正交矩阵与一个对角矩阵的乘积,通过两个核函数将两个正交矩阵列向量从原始非线性空间映射到一个高维特征空间.然后在低维特征空间中定义自适应正则化项,并将其与二维矩阵非线性方法整合于单个目标函数中,通过交替优化技术,在两个核子空间提取判别特征.最后在两个人脸数据集上的实验表明,文中算法在分类精度上获得较大提升.     

核不相关鉴别分析以及它在字符识别中的应用

核不相关鉴别分析是在线性不相关鉴别分析的基础上发展起来的．然而，由于核函数的运用，计算核不相关矢量集变得更加复杂．为了解决这个问题，提出一种解决核不相关鉴别分析的有效算法．该算法巧妙地利用了矩阵的分解，然后在一个矩阵对上进行广义奇异值分解．与此同时，提出了几个相关的定理．最重要的是，提出的算法能克服核不相关鉴别分析中矩阵的奇异问题．在某种意义上，提出的算法拓宽了已有的算法，即从线性问题到非线性问题．最后，用手写数字字符识别实验来验证提出的算法是可行和有效的．     

基于核正则化Fisher判据的故障诊断

Fisher判据分析(FDA)是工业过程故障诊断的常用降维算法,但其无法提取数据中的非线性关系。基于核的FDA(KFDA)是对FDA的非线性推广,但在FDA和KFDA小样本情况下都会面临矩阵的病态问题,正则化技术是解决该问题的有效途径。推导一种转化为方程组求解并表示成对偶形式的核正则化FDA算法,给出其用于故障诊断的流程,并针对TE过程的故障数据进行仿真。     

基于核空间中最优变换和聚类中心的鉴别特征提取

应用统计学习理论中的核化原理,可以将许多线性特征提取算法推广至非线性特征提取算法.提出了基于核化原理的最优变换与聚类中心算法,即通过非线性变换,将数据映射到高维核空间,应用最优变换算法,实现原空间数据的非线性特征提取,而求解过程却借助"核函数",回避了复杂非线性变换的具体表达形式.新算法可提取稳健的非线性鉴别特征,从而解决复杂分布数据的模式分类问题.大量数值实验表明新算法比传统的最优变换与聚类中心算法更有效,甚至优于经典的核Fisher判别分析.     

L_(2,1)范数正则化的广义核判别分析及其人脸识别

特征选取和子空间学习是人脸识别的关键问题。为更准确选取人脸中丰富的非线性特征,并解决小样本问题,提出了一种新的L_(2,1)范数正则化的广义核判别分析(generalized kernel discriminant analysis based on L_(2,1)-norm regularization,L21GKDA)。利用核函数将原始样本隐式地映射到高维特征空间中,得到广义核Fisher鉴别准则,再利用一种有效变换将该非线性模型转化为线性回归模型;为了能使特征选取和子空间学习同时进行,在模型中加入了一种L_(2,1)范数惩罚项,并给出该正则化方法的求解算法。因为方法借助于L_(2,1)范数惩罚项的特征选取能力,所以它能有效地提高识别率。在ORL、AR和PIE人脸库上的实验结果表明,新算法能有效选取人脸的非线性特征,提高判别能力。     

弹性网络核极限学习机的多标记学习算法

将正则化极限学习机或者核极限学习机理论应用到多标记分类中,一定程度上提高了算法的稳定性。但目前这些算法关于损失函数添加的正则项都基于L2正则,导致模型缺乏稀疏性表达。同时,弹性网络正则化既保证模型鲁棒性且兼具模型稀疏化学习,但结合弹性网络的极限学习机如何解决多标记问题鲜有研究。基于此,本文提出一种对核极限学习机添加弹性网络正则化的多标记学习算法。首先,对多标记数据特征空间使用径向基核函数映射;随后,对核极限学习机损失函数施加弹性网络正则项;最后,采用坐标下降法迭代求解输出权值以得到最终预测标记。通过对比试验和统计分析表明,提出的算法具有更好的性能表现。     

双边全变分的自适应核回归超分辨率重建

正则化方法是目前解决超分辨率重建中病态问题的一种被广泛使用的方法。在分析了现有基于多种正则化超分辨率重建方法的基础上,构造了一种基于双边全变分（BTV）的自适应核回归滤波核,并将它作为正则化超分辨率重构的代价函数,该方法根据图像特征自适应生成正则项的滤波核函数。实验结果表明,与传统的正则化重建方法相比较,该算法既能有效地去除噪声,也能很好地保留图像细节部分,同时还具有一定的鲁棒性。通过客观和主观评价表明,图像重建质量有显著的提高。     

利用组合核函数提高核主分量分析的性能

为了提高图像分类的识别率，在对基于核的学习算法中，核函数的构成条件以及不同核函数的特性进行分析和研究的基础上，提出了一种新的核函数——组合核函数，并将它应用于核主分量分析(KPCA)中，以便进行图像特征的提取，由于新的核函数既可以提取全局特征，又可以提取局部特征，因此，可以提高KPCA在图像特征提取中的性能。为了验证所提出核函数的有效性，首先利用新的核函数进行KPCA，以便对手写数字和脸谱等图像进行特征提取，然后利用线性支持向量机(SVM)来进行识别，实验结果显示，从识别率上看，用组合核函数所提取的特征质量比原核函数所提取的特征质量高。     

Face recognition using a kernel fractional-step discriminant analysis algorithm

Feature extraction is among the most important problems in face recognition systems. In this paper, we propose an enhanced kernel discriminant analysis (KDA) algorithm called kernel fractional-step discriminant analysis (KFDA) for nonlinear feature extraction and dimensionality reduction. Not only can this new algorithm, like other kernel methods, deal with nonlinearity required for many face recognition tasks, it can also outperform traditional KDA algorithms in resisting the adverse effects due to outlier classes. Moreover, to further strengthen the overall performance of KDA algorithms for face recognition, we propose two new kernel functions: cosine fractional-power polynomial kernel and non-normal Gaussian RBF kernel. We perform extensive comparative studies based on the YaleB and FERET face databases. Experimental results show that our KFDA algorithm outperforms traditional kernel principal component analysis (KPCA) and KDA algorithms. Moreover, further improvement can be obtained when the two new kernel functions are used.     

弹性核子空间聚类*

现有子空间聚类算法通常假设数据来自多个线性子空间,无法处理时间序列聚类中存在的非线性和时间轴弯曲问题.为了克服这些局限,通过引入核技巧和弹性距离,提出弹性核低秩表示子空间聚类和弹性核最小二乘回归子空间聚类,统称为弹性核子空间聚类,并从理论上证明弹性核最小二乘回归子空间算法的组效应和弹性核低秩表示子空间聚类算法的收敛性.在5个UCR时间序列数据集上的实验表明本文算法的有效性.     

On feature extraction via kernels.

Using the kernel trick idea and the kernels-as-features idea, we can construct two kinds of nonlinear feature spaces, where linear feature extraction algorithms can be employed to extract nonlinear features. In this correspondence, we study the relationship between the two kernel ideas applied to certain feature extraction algorithms such as linear discriminant analysis, principal component analysis, and canonical correlation analysis. We provide a rigorous theoretical analysis and show that they are equivalent up to different scalings on each feature. These results provide a better understanding of the kernel method.     

Kernel-Based State-Space Kriging for Predictive Control

In this paper, we extend the state-space kriging (SSK) modeling technique presented in a previous work by the authors in order to consider non-autonomous systems. SSK is a data-driven method that computes predictions as linear combinations of past outputs. To model the nonlinear dynamics of the system, we propose the kernel-based state-space kriging (K-SSK), a new version of the SSK where kernel functions are used instead of resorting to considerations about the locality of the data. Also, a Kalman filter can be used to improve the predictions at each time step in the case of noisy measurements. A constrained tracking nonlinear model predictive control (NMPC) scheme using the black-box input-output model obtained by means of the K-SSK prediction method is proposed. Finally, a simulation example and a real experiment are provided in order to assess the performance of the proposed controller.      

Fast kernel Fisher discriminant analysis via approximating the kernel principal component analysis

Kernel Fisher discriminant analysis (KFDA) extracts a nonlinear feature from a sample by calculating as many kernel functions as the training samples. Thus, its computational efficiency is inversely proportional to the size of the training sample set. In this paper we propose a more approach to efficient nonlinear feature extraction, FKFDA (fast KFDA). This FKFDA consists of two parts. First, we select a portion of training samples based on two criteria produced by approximating the kernel principal component analysis (AKPCA) in the kernel feature space. Then, referring to the selected training samples as nodes, we formulate FKFDA to improve the efficiency of nonlinear feature extraction. In FKFDA, the discriminant vectors are expressed as linear combinations of nodes in the kernel feature space, and the extraction of a feature from a sample only requires calculating as many kernel functions as the nodes. Therefore, the proposed FKFDA has a much faster feature extraction procedure compared with the naive kernel-based methods. Experimental results on face recognition and benchmark datasets classification suggest that the proposed FKFDA can generate well classified features.     

Distance difference and linear programming nonparallel plane classifier

We first propose Distance Difference GEPSVM (DGEPSVM), a binary classifier that obtains two nonparallel planes by solving two standard eigenvalue problems. Compared with GEPSVM, this algorithm does not need to care about the singularity occurring in GEPSVM, but with better classification correctness. This formulation is capable of dealing with XOR problems with different distribution for keeping the genuine geometrical interpretation of primal GEPSVM. Moreover, the proposed algorithm gives classification correctness comparable to that of LSTSVM and TWSVM, but with lesser unknown parameters. Then, the regularization techniques are incorporated to the TWSVM. With the help of the regularized formulation, a linear programming formation for TWSVM is proposed, called FETSVM, to improve TWSVM sparsity, thereby suppressing input features. This means FETSVM is capable of reducing the number of input features, for linear case. When a nonlinear classifier is used, this means few kernel functions determine the classifier. Lastly, this algorithm is compared on artificial and public datasets. To further illustrate the effectiveness of our proposed algorithms, we also apply these algorithms to USPS handwritten digits.