Fuzzy discriminant analysis with kernel methods

A novel fuzzy nonlinear classifier, called kernel fuzzy discriminant analysis (KFDA), is proposed to deal with linear non-separable problem. With kernel methods KFDA can perform efficient classification in kernel feature space. Through some nonlinear mapping the input data can be mapped implicitly into a high-dimensional kernel feature space where nonlinear pattern now appears linear. Different from fuzzy discriminant analysis (FDA) which is based on Euclidean distance, KFDA uses kernel-induced distance. Theoretical analysis and experimental results show that the proposed classifier compares favorably with FDA.     

Face recognition using a kernel fractional-step discriminant analysis algorithm

Feature extraction is among the most important problems in face recognition systems. In this paper, we propose an enhanced kernel discriminant analysis (KDA) algorithm called kernel fractional-step discriminant analysis (KFDA) for nonlinear feature extraction and dimensionality reduction. Not only can this new algorithm, like other kernel methods, deal with nonlinearity required for many face recognition tasks, it can also outperform traditional KDA algorithms in resisting the adverse effects due to outlier classes. Moreover, to further strengthen the overall performance of KDA algorithms for face recognition, we propose two new kernel functions: cosine fractional-power polynomial kernel and non-normal Gaussian RBF kernel. We perform extensive comparative studies based on the YaleB and FERET face databases. Experimental results show that our KFDA algorithm outperforms traditional kernel principal component analysis (KPCA) and KDA algorithms. Moreover, further improvement can be obtained when the two new kernel functions are used.     

Fault diagnosis of nonlinear and large-scale processes using novel modified kernel Fisher discriminant analysis approach

It is pretty significant for fault diagnosis timely and accurately to improve the dependability of industrial processes. In this study, fault diagnosis of nonlinear and large-scale processes by variable-weighted kernel Fisher discriminant analysis (KFDA) based on improved biogeography-based optimisation (IBBO) is proposed, referred to as IBBO-KFDA, where IBBO is used to determine the parameters of variable-weighted KFDA, and variable-weighted KFDA is used to solve the multi-classification overlapping problem. The main contributions of this work are four-fold to further improve the performance of KFDA for fault diagnosis. First, a nonlinear fault diagnosis approach with variable-weighted KFDA is developed for maximising separation between the overlapping fault samples. Second, kernel parameters and features selection of variable-weighted KFDA are simultaneously optimised using IBBO. Finally, a single fitness function that combines erroneous diagnosis rate with feature cost is created, a novel mixed kernel function is introduced to improve the classification capability in the feature space and diagnosis accuracy of the IBBO-KFDA, and serves as the target function in the optimisation problem. Moreover, an IBBO approach is developed to obtain the better quality of solution and faster convergence speed. On the one hand, the proposed IBBO-KFDA method is first used on Tennessee Eastman process benchmark data sets to validate the feasibility and efficiency. On the other hand, IBBO-KFDA is applied to diagnose faults of automation gauge control system. Simulation results demonstrate that IBBO-KFDA can obtain better kernel parameters and feature vectors with a lower computing cost, higher diagnosis accuracy and a better real-time capacity.     

一种基于空间变换的核Fisher鉴别分析

引入空间变换的思相想，提出了一种基于空间变换的核Fisher鉴别分析，与KFDA不同的是，该方法只需在一个较低维的空间内执行，从而较大幅度地降低了求解最优鉴别矢量集的计算量，提高了计算速度，在ORL标准人脸库上的试验结果验证了所提方法的有效性。     

一种用于人脸识别的非线性鉴别特征融合方法

最近，在人脸等图像识别领域，用于抽取非线性特征的核方法如核Fisher鉴别分析(KFDA)已经取得成功并得到了广泛应用，但现有的核方法都存在这样的问题，即构造特征空间中的核矩阵所耗费的计算量非常大．而且，抽取得到的单类特征往往不能获得到令人满意的识别结果．提出了一种用于人脸识别的非线性鉴别特征融合方法，即首先利用小波变换和奇异值分解对原始输入样本进行降雏变换，抽取同一样本空间的两类特征，然后利用复向量将这两类特征组合在一起，构成一复特征向量空间，最后在该空间中进行最优鉴别特征抽取．在ORL标准人脸库上的试验结果表明所提方法不仅在识别性能上优于现有的核Fisher鉴别分析方法，而且，在ORL人脸库上的特征抽取速度提高了近8倍．     

Improvement of the kernel minimum squared error model for fast feature extraction

The kernel minimum squared error (KMSE) expresses the feature extractor as a linear combination of all the training samples in the high-dimensional kernel space. To extract a feature from a sample, KMSE should calculate as many kernel functions as the training samples. Thus, the computational efficiency of the KMSE-based feature extraction procedure is inversely proportional to the size of the training sample set. In this paper, we propose an efficient kernel minimum squared error (EKMSE) model for two-class classification. The proposed EKMSE expresses each feature extractor as a linear combination of nodes, which are a small portion of the training samples. To extract a feature from a sample, EKMSE only needs to calculate as many kernel functions as the nodes. As the nodes are commonly much fewer than the training samples, EKMSE is much faster than KMSE in feature extraction. The EKMSE can achieve the same training accuracy as the standard KMSE. Also, EKMSE avoids the overfitting problem. We implement the EKMSE model using two algorithms. Experimental results show the feasibility of the EKMSE model.     

An efficient renovation on kernel Fisher discriminant analysis and face recognition experiments

A reformative kernel algorithm, which can deal with two-class problems as well as those with more than two classes, on Fisher discriminant analysis is proposed. In the novel algorithm the supposition that in feature space discriminant vector can be approximated by some linear combination of a part of training samples, called “significant nodes”, is made. If the “significant nodes” are found out, the novel algorithm on kernel Fisher discriminant analysis will be superior to the naive one in classification efficiency. In this paper, a recursive algorithm for selecting “significant nodes”, is developed in detail. Experiments show that the novel algorithm is effective and much efficient in classifying.     

基于核Fisher判别分析的蛋白质氧链糖基化位点的预测

以各种窗口长度的蛋白质样本序列为研究对象,实验样本用稀疏编码方式编码,使用核Fisher判别分析(KFDA)的方法来预测蛋白质氧链糖基化位点。首先通过非线性映射(由核函数隐含定义)将样本映射到特征空间,然后在特征空间中用Fisher判别分析进行分类。进一步,用多数投票策略对各种窗口下的分类器进行组合以综合多个窗口的优势。实验结果表明,使用组合KFDA的方法预测的效果优于FDA和PCA以及单个KFDA分类器的预测效果,预测准确率为86.5%。     

基于低密度分割几何距离的半监督KFDA算法

提出了一种基于低密度分割几何距离的半监督KFDA（kernel Fisher discriminant analysis）算法（semisupervised KFDA based on low density separation geometry distance，简称SemiGKFDA）.该算法以低密度分割几何距离作为相似性度量，通过大量无标签样本，提高KFDA算法的泛化能力.首先，利用核函数将原始空间样本数据映射到高维特征空间中；然后，通过有标签样本和无标签样本构建低密度分割几何距离测度上的内蕴结构一致性假设，使其作为正则化项整合到费舍尔判别分析的目标函数中；最后，通过求解最小化目标函数获得最优投影矩阵.人工数据集和UCI数据集上的实验表明，该算法与KFDA及其改进算法相比，在分类性能上有显著提高.此外，将该算法与其他算法应用到人脸识别问题中进行对比，实验结果表明，该算法具有更高的识别精度.     

A fast kernel-based nonlinear discriminant analysis for multi-class problems

Nonlinear discriminant analysis may be transformed into the form of kernel-based discriminant analysis. Thus, the corresponding discriminant direction can be solved by linear equations. From the view of feature space, the nonlinear discriminant analysis is still a linear method, and it is provable that in feature space the method is equivalent to Fisher discriminant analysis. We consider that one linear combination of parts of training samples, called “significant nodes”, can replace the total training samples to express the corresponding discriminant vector in feature space to some extent. In this paper, an efficient algorithm is proposed to determine “significant nodes” one by one. The principle of determining “significant nodes” is simple and reasonable, and the consequent algorithm can be carried out with acceptable computation cost. Depending on the kernel functions between test samples and all “significant nodes”, classification can be implemented. The proposed method is called fast kernel-based nonlinear method (FKNM). It is noticeable that the number of “significant nodes” may be much smaller than that of the total training samples. As a result, for two-class classification problems, the FKNM will be much more efficient than the naive kernel-based nonlinear method (NKNM). The FKNM can be also applied to multi-class via two approaches: one-against-the-rest and one-against-one. Although there is a view that one-against-one is superior to one-against-the-rest in classification efficiency, it seems that for the FKNM one-against-the-rest is more efficient than one-against-one. Experiments on benchmark and real datasets illustrate that, for two-class and multi-class classifications, the FKNM is effective, feasible and much efficient.     

局部Fisher判别分析算法及仿真研究*

复杂化工过程常被多种类型的故障损坏,正常的训练数据无法建立准确的操作模型。为了提高复杂化工过程中故障的检测和分类能力,传统无监督Fisher判别分析(Fisher Discriminant Analysis,FDA)算法无法在多模态故障数据中的应用,本文提出基于局部Fisher判别分析(Local Fisher Discriminant Analysis,LFDA)的故障诊断方法。首先计算训练数据的局部类内和类间离散度矩阵,寻找LFDA的投影方向;其次把训练数据和测试数据向投影向量上投影,提取特征向量;最后计算特征向量间的欧氏距离,运用KNN分类器进行分类。把提出的LFDA方法应用到Tennessee Eastman(TE)过程,监控结果表明,LFDA的效果好于FDA和核Fisher判别分析(Kernel Fisher Discriminant Analysis,KFDA),说明LFDA方法在分类及检测不同类的故障方面具有高准确性及高灵敏度的优势。     

Generalized nonlinear discriminant analysis and its small sample size problems

This paper develops a generalized nonlinear discriminant analysis (GNDA) method and deals with its small sample size (SSS) problems. GNDA is a nonlinear extension of linear discriminant analysis (LDA), while kernel Fisher discriminant analysis (KFDA) can be regarded as a special case of GNDA. In LDA, an under sample problem or a small sample size problem occurs when the sample size is less than the sample dimensionality, which will result in the singularity of the within-class scatter matrix. Due to a high-dimensional nonlinear mapping in GNDA, small sample size problems arise rather frequently. To tackle this issue, this research presents five different schemes for GNDA to solve the SSS problems. Experimental results on real-world data sets show that these schemes for GNDA are very effective in tackling small sample size problems.     

A reformative kernel Fisher discriminant analysis

A reformative kernel Fisher discriminant method is proposed, which is directly derived from the naive kernel Fisher discriminant analysis with superiority in classification efficiency. In the novel method only a part of training patterns, called “significant nodes”, are necessary to be adopted in classifying one test pattern. A recursive algorithm for selecting “significant nodes”, which is the key of the novel method, is presented in detail. The experiment on benchmarks shows that the novel method is effective and much efficient in classifying.     

鉴别分析方法的最优化问题研究

提出了一种核Fisher鉴别分析方法优化方案,并分别给出了解决两类分类和解决多于两类的分类问题的算法,该方案具有明显的分类效率上的优势。在这种方案的实现中,首先从总体训练样本中选择出“显著”训练样本,对测试样本的分类只依赖于测试样本与“显著”训练样本之间的核函数。还设计出了一种选择“显著”训练样本的递归算法,以降低算法的计算复杂度。将该算法应用于人脸图象数据库与“基准”数据集,得到了很好的实验效果。     

基于核正交半监督鉴别分析的人脸识别算法

针对人脸识别中的非线性特征提取和有标记样本不足问题,提出了在核空间具有正交性半监督鉴别矢量的计算方法。算法利用核函数将人脸数据映射到高维非线性空间,在该空间采用边界Fisher判别分析（Marginal Fisher Analysis,MFA）算法将少量有类别标签样本进行降维,同时采用无监督鉴别投影（Unsupervised Discriminant Projection,UDP）对大量无标签样本进行学习,以半监督的方法构造算法的目标函数,在特征值求解时以正交方式找出最优投影向量,进行人脸识别。通过实验,在ORL和YALE人脸数据库上验证了该算法的有效性。     

Essence of kernel Fisher discriminant: KPCA plus LDA

In this paper, the method of kernel Fisher discriminant (KFD) is analyzed and its nature is revealed, i.e., KFD is equivalent to kernel principal component analysis (KPCA) plus Fisher linear discriminant analysis (LDA). Based on this result, a more transparent KFD algorithm is proposed. That is, KPCA is first performed and then LDA is used for a second feature extraction in the KPCA-transformed space. Finally, the effectiveness of the proposed algorithm is verified using the CENPARMI handwritten numeral database.     

Fault diagnosis and classification framework using multi-scale classification based on kernel Fisher discriminant analysis for chemical process system

Fault detection and diagnosis (FDD) in chemical process systems is an important tool for effective process monitoring to ensure the safety of a process. Multi-scale classification offers various advantages for monitoring chemical processes generally driven by events in different time and frequency domains. However, there are issues when dealing with highly interrelated, complex, and noisy databases with large dimensionality. Therefore, a new method for the FDD framework is proposed based on wavelet analysis, kernel Fisher discriminant analysis (KFDA), and support vector machine (SVM) classifiers. The main objective of this work was to combine the advantages of these tools to enhance the performance of the diagnosis on a chemical process system. Initially, a discrete wavelet transform (DWT) was applied to extract the dynamics of the process at different scales. The wavelet coefficients obtained during the analysis were reconstructed using the inverse discrete wavelet transform (IDWT) method, which were then fed into the KFDA to produce discriminant vectors. Finally, the discriminant vectors were used as inputs for the SVM classification task. The SVM classifiers were utilized to classify the feature sets extracted by the proposed method. The performance of the proposed multi-scale KFDA-SVM method for fault classification and diagnosis was analysed and compared using a simulated Tennessee Eastman process as a benchmark. The results showed the improvements of the proposed multiscale KFDA-SVM framework with an average 96.79% of classification accuracy over the multi-scale KFDA-GMM (84.94%), and the established independent component analysis-SVM method (95.78%) of the faults in the Tennessee Eastman process.     

核参数判别选择方法在核主元分析中的应用

针对核主元分析(KPCA)中高斯核参数β的经验选取问题,提出了核主元分析的核参数判别选择方法。依据训练样本的类标签计算类内、类间核窗宽,在以上核窗宽中经判别选择方法确定核参数。根据判别选择核参数所确定的核矩阵,能够准确描述训练空间的结构特征。用主成分分析(PCA)对特征空间进行分解,提取主成分以实现降维和特征提取。判别核窗宽方法在分类密集区域选择较小窗宽,在分类稀疏区域选择较大窗宽。将判别核主成分分析(Dis-KPCA)应用到数据模拟实例和田纳西过程(TEP),通过与KPCA、PCA方法比较,实验结果表明,Dis-KPCA方法有效地对样本数据降维且将三个类别数据100%分开,因此,所提方法的降维精度更高。     

人脸识别中基于核的子空间鉴别分析

尽管基于Fisher准则的线性鉴别分析被公认为特征抽取的有效方法之一，并被成功地用于人脸识别，但是由于光照变化、人脸表情和姿势变化，实际上的人脸图像分布是十分复杂的，因此，抽取非线性鉴别特征显得十分必要。为了能利用非线性鉴别特征进行人脸识别，提出了一种基于核的子空间鉴别分析方法。该方法首先利用核函数技术将原始样本隐式地映射到高维（甚至无穷维）特征空间；然后在高维特征空间里，利用再生核理论来建立基于广义Fisher准则的两个等价模型；最后利用正交补空间方法求得最优鉴别矢量来进行人脸识别。在ORL和NUST603两个人脸数据库上，对该方法进行了鉴别性能实验，得到了识别率分别为94％和99．58％的实验结果，这表明该方法与核组合方法的识别结果相当，且明显优于KPCA和Kernel fisherfaces方法的识别结果。     

基于核方法的雷达目标一维距离像识别

由于雷达目标及其所处环境的复杂性,导致不同目标之间的关系往往是非线性的.研究基于核的非线性方法,并将其应用于雷达目标一维距离像识别.核Fisher判别分析(KFDA)是一种抽取非线性特征的最有效方法之一,但它往往会面临小样本问题.针对此问题,给出一种null-KFDA方法,对距离像进行特征提取.然后,采用一种新的核非线性分类器——KNR(kernel-based nonlinear representor),对所提取的特征进行分类.对3种飞机的实测距离像进行实验,结果验证了null-KFDA的有效性.此外,与非线性支持向量机(SVM)和径向基函数神经网络(RBFNN)相比,KNR分类器具有更优的识别性能.