FKA算法迭代收敛性分析

《核选择和非线性特征提取的双线性分析》一文提出了一种新颖的核Fisher准则FKC, 并用迭代分析算法FKA求得最优解,但其迭代收敛性缺乏理论上的证明。从理论上对FKA算法的迭代收敛性进行了分析和探讨,并运用Radermacher复杂性分析法进行证明。     

Kernel-based improved discriminant analysis and its application to face recognition

Kernel discriminant analysis (KDA) is a widely used tool in feature extraction community. However, for high-dimensional multi-class tasks such as face recognition, traditional KDA algorithms have the limitation that the Fisher criterion is nonoptimal with respect to classification rate. Moreover, they suffer from the small sample size problem. This paper presents a variant of KDA called kernel-based improved discriminant analysis (KIDA), which can effectively deal with the above two problems. In the proposed framework, origin samples are projected firstly into a feature space by an implicit nonlinear mapping. After reconstructing between-class scatter matrix in the feature space by weighted schemes, the kernel method is used to obtain a modified Fisher criterion directly related to classification error. Finally, simultaneous diagonalization technique is employed to find lower-dimensional nonlinear features with significant discriminant power. Experiments on face recognition task show that the proposed method is superior to the traditional KDA and LDA.     

基于核的Fisher非线性最佳鉴别分析在人脸识别中的应用

抽取最佳鉴别特征是人脸识别中的重要一步。对小样本的高维人脸图像样本,由于各种抽取非线性鉴别特征的方法均存在各自的问题,为此提出了一种求解核的Fisher非线性最佳鉴别特征的新方法,该方法首先在特征空间用类间散度阵和类内散度阵作为Fisher准则,来得到最佳非线性鉴别特征,然后针对此方法存在的病态问题,进一步在类内散度阵的零空间中求解最佳非线性鉴别矢量。基于ORL人脸数据库的实验表明,该新方法抽取的非线性最佳鉴别特征明显优于Fisher线性鉴别分析(FLDA)的线性特征和广义鉴别分析(GDA)的非线性特征。     

人脸识别中基于核的子空间鉴别分析

尽管基于Fisher准则的线性鉴别分析被公认为特征抽取的有效方法之一，并被成功地用于人脸识别，但是由于光照变化、人脸表情和姿势变化，实际上的人脸图像分布是十分复杂的，因此，抽取非线性鉴别特征显得十分必要。为了能利用非线性鉴别特征进行人脸识别，提出了一种基于核的子空间鉴别分析方法。该方法首先利用核函数技术将原始样本隐式地映射到高维（甚至无穷维）特征空间；然后在高维特征空间里，利用再生核理论来建立基于广义Fisher准则的两个等价模型；最后利用正交补空间方法求得最优鉴别矢量来进行人脸识别。在ORL和NUST603两个人脸数据库上，对该方法进行了鉴别性能实验，得到了识别率分别为94％和99．58％的实验结果，这表明该方法与核组合方法的识别结果相当，且明显优于KPCA和Kernel fisherfaces方法的识别结果。     

Face recognition using kernel scatter-difference-based discriminant analysis

There are two fundamental problems with the Fisher linear discriminant analysis for face recognition. One is the singularity problem of the within-class scatter matrix due to small training sample size. The other is that it cannot efficiently describe complex nonlinear variations of face images because of its linear property. In this letter, a kernel scatter-difference-based discriminant analysis is proposed to overcome these two problems. We first use the nonlinear kernel trick to map the input data into an implicit feature space F. Then a scatter-difference-based discriminant rule is defined to analyze the data in F. The proposed method can not only produce nonlinear discriminant features but also avoid the singularity problem of the within-class scatter matrix. Extensive experiments show encouraging recognition performance of the new algorithm.     

一种改进的核特征抽取方法及其在人脸识别中的应用

首先利用核函数技术将原始样本隐式地映射到高维特征空间;然后在高维空间里利用再生核理论建立基于Fisher鉴别极小准则的2个等价模型;最后在该空间的核类间散布矩阵的非零空间和零空间中应用Fisher极小鉴别准则求取核鉴别矢量.在人脸库上的实验结果验证了该算法的有效性.     

基于核费希尔判别的生产状态诊断方法

工艺参数间的非线性耦合关系,给生产过程的状态识别带来了很大的困难。为此,引入新的核映射准则,利用梯度优化方法选取核参数,并采用核Fisher方法进行降维处理,实现对生产状态在可视平面上的逐层多故障分类,完成对当前生产过程的状态诊断。利用TE数据进行实验验证,结果表明,与核主成分分析方法相比,该方法可以得到更加准确的诊断结果。     

基于Bhattacharyya距离准则的核空间特征提取算法

提出了一种新的以Bhattacharyya距离为准则的核空间特征提取算法．该算法的核心思想是把样本非线性映射到高维核空间．在核空间中寻找一组最优特征向量，然后把样本线性映射到低维特征空间，使类别间的Bhattacharyya距离最大。从而保证Bayes分类误差上界最小．采用核函数技术，把特征提取问题转化为一个QP(Quadratic Programming)优化问题．保证了算法的全局收敛性和快速性．此算法具有两个优点：(1)该算法提取的特征对数据分类来说更有效；(2)对于给定的模式分类问题，算法可以预测出在不损失分类精度情况下所必须的特征向量数目的上界，并能够提取出分类有效特征．实验结果表明，该算法的性能与理论分析的结论相吻合，优于目前常用的特征提取算法．     

基于核的多元区别分析算法的特征抽取及其在人脸识别中的应用

提出了一种基于核技术的求多元区别分析最佳解的K1PMDA算法，并把这一算法应用于人脸识别中。对线性人脸识别中存在两个突出问题：1、在光照、表情、姿态变化较大时，人脸图像分类是复杂的、非线性的；2、小样本问题，即当训练样本数量小于样本特征空间维数时，导致类内散布矩阵奇异。对于前一个问题，可以采用核技术提取人脸图像样本的非线性特征，对于后一个问题，采用加入一个扰动参数的扰动算法。通过对ORL，Yale Group B以及UMIST三个人脸库的实验表明，该算法是可行的、高效的。     

基于KIOFD算法的特征抽取及其在人脸识别中的应用

提出了一种基于核技术的融合了反转Fisher鉴别准则和正交化技术的KIOFD（Kernel Inverse Orthogonalized Fisher Discriminant）算法,并把这一算法应用于人脸识别中。线性人脸识别中存在两个突出问题:（1）在光照、表情、姿态变化较大时,人脸图像分类是复杂的、非线性的;（2）小样本问题,即当训练样本数量小于样本特征空间维数时,导致类内散布矩阵奇异。对于第1个问题,可以采用核技术提取人脸图像样本的非线性特征,对于第2个问题,采用了反转Fisher鉴别准则和正交化结合的算法。通过对ORL、Yale Group B以及UMIST3个人脸库的实验表明,提出的算法是可行的、高效的。     

Generalized nonlinear discriminant analysis and its small sample size problems

This paper develops a generalized nonlinear discriminant analysis (GNDA) method and deals with its small sample size (SSS) problems. GNDA is a nonlinear extension of linear discriminant analysis (LDA), while kernel Fisher discriminant analysis (KFDA) can be regarded as a special case of GNDA. In LDA, an under sample problem or a small sample size problem occurs when the sample size is less than the sample dimensionality, which will result in the singularity of the within-class scatter matrix. Due to a high-dimensional nonlinear mapping in GNDA, small sample size problems arise rather frequently. To tackle this issue, this research presents five different schemes for GNDA to solve the SSS problems. Experimental results on real-world data sets show that these schemes for GNDA are very effective in tackling small sample size problems.     

Optimizing the data-dependent kernel under a unified kernel optimization framework

The kernel functions play a central role in kernel methods, accordingly over the years the optimization of kernel functions has been a promising research area. Ideally Fisher discriminant criteria can be used as an objective function to optimize the kernel function to augment the margin between different classes. Unfortunately, Fisher criteria are optimal only in the case that all the classes are generated from underlying multivariate normal distributions of common covariance matrix but different means and each class is expressed by a single cluster. Due to the assumptions, Fisher criteria obviously are not a suitable choice as a kernel optimization rule in some applications such as the multimodally distributed data. In order to solve this problem, recently many improved discriminant criteria (DC) have been also developed. Therefore, to apply these discriminant criteria to kernel optimization, in this paper based on a data-dependent kernel function we propose a unified kernel optimization framework, which can use any discriminant criteria formulated in a pairwise manner as the objective functions. Under the kernel optimization framework, to employ different discriminant criteria, one has to only change the corresponding affinity matrices without having to resort to any complex derivations in feature space. Experimental results based on some benchmark data demonstrate the efficiency of our method.     

KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition

This paper examines the theory of kernel Fisher discriminant analysis (KFD) in a Hilbert space and develops a two-phase KFD framework, i.e., kernel principal component analysis (KPCA) plus Fisher linear discriminant analysis (LDA). This framework provides novel insights into the nature of KFD. Based on this framework, the authors propose a complete kernel Fisher discriminant analysis (CKFD) algorithm. CKFD can be used to carry out discriminant analysis in "double discriminant subspaces." The fact that, it can make full use of two kinds of discriminant information, regular and irregular, makes CKFD a more powerful discriminator. The proposed algorithm was tested and evaluated using the FERET face database and the CENPARMI handwritten numeral database. The experimental results show that CKFD outperforms other KFD algorithms.     

基于多核Fisher判别分析的人脸特征提取

核Fisher判别分析法是一种有效的非线性判别分析法。传统的核Fisher判别分析仅选用单个核函数,在人脸特征提取方面仍显不足。鉴于此,提出多核Fisher判别分析法,即通过将多个单核Fisher判别得到的投影进行加权组合得到加权投影,以加权投影为依据进行特征提取和分类。实验表明,在进行人脸特征提取和分类时,多核Fisher判别分析法优于单核Fisher判别分析法。     

子空间半监督Fisher判别分析

Fisher判别分析寻找一个使样本数据类间散度与样本数据类内散度比值最大的子空间, 是一种很流行的监督式特征降维方法. 标注样本数据所属的类别通常需要大量的人工, 消耗大量的时间, 付出昂贵的成本. 为了解决同时利用有类别信息的样本数据和没有类别信息的样本数据用于寻找降维子空间的问题, 我们提出了一种子空间半监督Fisher判别分析方法. 子空间半监督Fisher判别分析寻找这样一个子空间, 这个子空间即保留了从有类别信息的样本数据中学习的类别判别结构, 也保留了从有类别信息的样本数据和没有类别信息的样本数据中学习的样本结构信息. 我们还推导了基于核的子空间半监督Fisher判别分析方法. 通过人脸识别实验验证了本文算法的有效性.     

核诱导距离度量的鲁棒判别分析

提出了基于核诱导距离度量的鲁棒判别分析算法(robust discriminant analysis based on kernel-induced distance measure,KI-RDA)。KI-RDA不仅自然地推广了线性判别分析(linear discriminant analysis,LDA),而且推广了最近提出的强有力的基于非参数最大熵的鲁棒判别分析(robust discriminant analysis based on nonparametric maximum entropy,MaxEnt-RDA)。通过采用鲁棒径向基核,KI-RDA不仅能有效处理含噪数据,而且也适合处理非高斯分布的非线性数据,其本质的鲁棒性归咎于KI-RDA通过核诱导的非欧距离代替LDA的欧氏距离来刻画类间散度和类内散度。借助这些散度,为特征提取定义类似LDA的判别准则,导致了相应的非线性优化问题。进一步借助近似策略,将优化问题转化为直接可解的广义特征值问题,由此获得降维变换(矩阵)的闭合解。最后在多类数据集上进行实验,验证了KI-RDA的有效性。由于核的多样性,使KI-RDA事实上成为了一个一般性判别分析框架。     

A kernel optimization method based on the localized kernel Fisher criterion

It is widely recognized that whether the selected kernel matches the data determines the performance of kernel-based methods. Ideally it is expected that the data is linearly separable in the kernel induced feature space, therefore, Fisher linear discriminant criterion can be used as a cost function to optimize the kernel function. However, the data may not be linearly separable even after kernel transformation in many applications, e.g., the data may exist as multimodally distributed structure, in this case, a nonlinear classifier is preferred, and obviously Fisher criterion is not a suitable choice as kernel optimization rule. Motivated by this issue, we propose a localized kernel Fisher criterion, instead of traditional Fisher criterion, as the kernel optimization rule to increase the local margins between embedded classes in kernel induced feature space. Experimental results based on some benchmark data and measured radar high-resolution range profile (HRRP) data show that the classification performance can be improved by using the proposed method.     

基于核函数的Fisher判别分析算法在人耳识别中的应用

针对人耳图像自身的特点,并通过对现有生物识别技术的研究,本文尝试采用了一种基于核函数的Fisher判别分析算法对人耳进行识别。该算法不仅可以有效地提取人耳特征,获得较高的识别率;而且还可以解决因为光照和人耳旋转角度等因素带来的非线性问题。实验表明:采用基于径向基核函数的Fisher判别分析算法对人耳图像进行识别,其识别率最高,为98.701%。     

Breast cancer diagnosis using genetic programming generated feature

This paper proposes a novel method for breast cancer diagnosis using the feature generated by genetic programming (GP). We developed a new feature extraction measure (modified Fisher linear discriminant analysis (MFLDA)) to overcome the limitation of Fisher criterion. GP as an evolutionary mechanism provides a training structure to generate features. A modified Fisher criterion is developed to help GP optimize features that allow pattern vectors belonging to different categories to distribute compactly and disjoint regions. First, the MFLDA is experimentally compared with some classical feature extraction methods (principal component analysis, Fisher linear discriminant analysis, alternative Fisher linear discriminant analysis). Second, the feature generated by GP based on the modified Fisher criterion is compared with the features generated by GP using Fisher criterion and an alternative Fisher criterion in terms of the classification performance. The classification is carried out by a simple classifier (minimum distance classifier). Finally, the same feature generated by GP is compared with a original feature set as the inputs to multi-layer perceptrons and support vector machine. Results demonstrate the capability of this method to transform information from high-dimensional feature space into one-dimensional space and automatically discover the relationship among data, to improve classification accuracy.     

基于核Fisher判别分析视频运动目标的分类

针对线性判别分析只能提取线性特征而不能描述非线性特征的缺点,采用将核函数和 Fisher判别分析方法的可分性结合起来的核 Fisher判别分析的方法对视频中的运动目标进行自动分类,运动目标包含人、汽车和宠物三类。该方法取得了较好的分类效果,且在查全率、查准率和 F1-Measure 获得了满意的性能。