A reformative kernel Fisher discriminant analysis

A reformative kernel Fisher discriminant method is proposed, which is directly derived from the naive kernel Fisher discriminant analysis with superiority in classification efficiency. In the novel method only a part of training patterns, called “significant nodes”, are necessary to be adopted in classifying one test pattern. A recursive algorithm for selecting “significant nodes”, which is the key of the novel method, is presented in detail. The experiment on benchmarks shows that the novel method is effective and much efficient in classifying.     

A fast kernel-based nonlinear discriminant analysis for multi-class problems

Nonlinear discriminant analysis may be transformed into the form of kernel-based discriminant analysis. Thus, the corresponding discriminant direction can be solved by linear equations. From the view of feature space, the nonlinear discriminant analysis is still a linear method, and it is provable that in feature space the method is equivalent to Fisher discriminant analysis. We consider that one linear combination of parts of training samples, called “significant nodes”, can replace the total training samples to express the corresponding discriminant vector in feature space to some extent. In this paper, an efficient algorithm is proposed to determine “significant nodes” one by one. The principle of determining “significant nodes” is simple and reasonable, and the consequent algorithm can be carried out with acceptable computation cost. Depending on the kernel functions between test samples and all “significant nodes”, classification can be implemented. The proposed method is called fast kernel-based nonlinear method (FKNM). It is noticeable that the number of “significant nodes” may be much smaller than that of the total training samples. As a result, for two-class classification problems, the FKNM will be much more efficient than the naive kernel-based nonlinear method (NKNM). The FKNM can be also applied to multi-class via two approaches: one-against-the-rest and one-against-one. Although there is a view that one-against-one is superior to one-against-the-rest in classification efficiency, it seems that for the FKNM one-against-the-rest is more efficient than one-against-one. Experiments on benchmark and real datasets illustrate that, for two-class and multi-class classifications, the FKNM is effective, feasible and much efficient.     

一种基于核的快速非线性鉴别分析方法

基于“核技巧”提出的新的非线性鉴别分析方法在最小二乘意义上与基于核的Fisher鉴别分析方法等效,相应鉴别方向通过一个线性方程组得出,计算代价较小,相应分类实现极其简便．该方法的最大优点是,对训练数据进行筛选,可使构造鉴别矢量的“显著”训练模式数大大低于总训练模式数,从而使得测试集的分类非常高效;同时,设计出专门的优化算法以加速“显著”训练模式的选取．实验表明,这种非线性方法不仅具有明显的效率上的优势,且具有不低于基于核的Fisher鉴别分析方法的性能．     

KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition

This paper examines the theory of kernel Fisher discriminant analysis (KFD) in a Hilbert space and develops a two-phase KFD framework, i.e., kernel principal component analysis (KPCA) plus Fisher linear discriminant analysis (LDA). This framework provides novel insights into the nature of KFD. Based on this framework, the authors propose a complete kernel Fisher discriminant analysis (CKFD) algorithm. CKFD can be used to carry out discriminant analysis in "double discriminant subspaces." The fact that, it can make full use of two kinds of discriminant information, regular and irregular, makes CKFD a more powerful discriminator. The proposed algorithm was tested and evaluated using the FERET face database and the CENPARMI handwritten numeral database. The experimental results show that CKFD outperforms other KFD algorithms.     

Bilinear analysis for kernel selection and nonlinear feature extraction.

This paper presents a unified criterion, Fisher + kernel criterion (FKC), for feature extraction and recognition. This new criterion is intended to extract the most discriminant features in different nonlinear spaces, and then, fuse these features under a unified measurement. Thus, FKC can simultaneously achieve nonlinear discriminant analysis and kernel selection. In addition, we present an efficient algorithm Fisher + kernel analysis (FKA), which utilizes the bilinear analysis, to optimize the new criterion. This FKA algorithm can alleviate the ill-posed problem existed in traditional kernel discriminant analysis (KDA), and usually, has no singularity problem. The effectiveness of our proposed algorithm is validated by a series of face-recognition experiments on several different databases.     

人脸识别中基于核的子空间鉴别分析

尽管基于Fisher准则的线性鉴别分析被公认为特征抽取的有效方法之一，并被成功地用于人脸识别，但是由于光照变化、人脸表情和姿势变化，实际上的人脸图像分布是十分复杂的，因此，抽取非线性鉴别特征显得十分必要。为了能利用非线性鉴别特征进行人脸识别，提出了一种基于核的子空间鉴别分析方法。该方法首先利用核函数技术将原始样本隐式地映射到高维（甚至无穷维）特征空间；然后在高维特征空间里，利用再生核理论来建立基于广义Fisher准则的两个等价模型；最后利用正交补空间方法求得最优鉴别矢量来进行人脸识别。在ORL和NUST603两个人脸数据库上，对该方法进行了鉴别性能实验，得到了识别率分别为94％和99．58％的实验结果，这表明该方法与核组合方法的识别结果相当，且明显优于KPCA和Kernel fisherfaces方法的识别结果。     

子空间半监督Fisher判别分析

Fisher判别分析寻找一个使样本数据类间散度与样本数据类内散度比值最大的子空间, 是一种很流行的监督式特征降维方法. 标注样本数据所属的类别通常需要大量的人工, 消耗大量的时间, 付出昂贵的成本. 为了解决同时利用有类别信息的样本数据和没有类别信息的样本数据用于寻找降维子空间的问题, 我们提出了一种子空间半监督Fisher判别分析方法. 子空间半监督Fisher判别分析寻找这样一个子空间, 这个子空间即保留了从有类别信息的样本数据中学习的类别判别结构, 也保留了从有类别信息的样本数据和没有类别信息的样本数据中学习的样本结构信息. 我们还推导了基于核的子空间半监督Fisher判别分析方法. 通过人脸识别实验验证了本文算法的有效性.     

基于低密度分割几何距离的半监督KFDA算法

提出了一种基于低密度分割几何距离的半监督KFDA（kernel Fisher discriminant analysis）算法（semisupervised KFDA based on low density separation geometry distance，简称SemiGKFDA）.该算法以低密度分割几何距离作为相似性度量，通过大量无标签样本，提高KFDA算法的泛化能力.首先，利用核函数将原始空间样本数据映射到高维特征空间中；然后，通过有标签样本和无标签样本构建低密度分割几何距离测度上的内蕴结构一致性假设，使其作为正则化项整合到费舍尔判别分析的目标函数中；最后，通过求解最小化目标函数获得最优投影矩阵.人工数据集和UCI数据集上的实验表明，该算法与KFDA及其改进算法相比，在分类性能上有显著提高.此外，将该算法与其他算法应用到人脸识别问题中进行对比，实验结果表明，该算法具有更高的识别精度.     

Perturbation LDA: Learning the difference between the class empirical mean and its expectation

Fisher's linear discriminant analysis (LDA) is popular for dimension reduction and extraction of discriminant features in many pattern recognition applications, especially biometric learning. In deriving the Fisher's LDA formulation, there is an assumption that the class empirical mean is equal to its expectation. However, this assumption may not be valid in practice. In this paper, from the “perturbation” perspective, we develop a new algorithm, called perturbation LDA (P-LDA), in which perturbation random vectors are introduced to learn the effect of the difference between the class empirical mean and its expectation in Fisher criterion. This perturbation learning in Fisher criterion would yield new forms of within-class and between-class covariance matrices integrated with some perturbation factors. Moreover, a method is proposed for estimation of the covariance matrices of perturbation random vectors for practical implementation. The proposed P-LDA is evaluated on both synthetic data sets and real face image data sets. Experimental results show that P-LDA outperforms the popular Fisher's LDA-based algorithms in the undersampled case.     

Face recognition based on 2D Fisherface approach

Two-dimensional (2D) discrimination analysis using methods such as 2D PCA and Image LDA is of interest in face recognition because it extracts discriminative features faster than one-dimensional (1D) discrimination analysis. However, existing 2D methods generally use more discriminative features and take longer to test than 1D methods. 2D PCA in particular cannot make full use of the Fisher discriminant criterion. Image LDA also has drawbacks in that it cannot perform 2D principal component analysis and discards components with poor discriminative capabilities. In addition, existing 2D methods cannot provide an automatic strategy to choose 2D principal components or discriminant vectors. In this paper, we propose 2D Fisherface, a novel discrimination approach that combines the two-stage “PCA+LDA” strategy and 2D discrimination techniques. It can extract face discriminative features by automatically selecting two-dimensional principal components and discriminant vectors. Using the AR database as the test data, it is shown that the proposed approach is faster and more effective than several representative 1D and 2D discrimination methods.     

基于多核Fisher判别分析的人脸特征提取

核Fisher判别分析法是一种有效的非线性判别分析法。传统的核Fisher判别分析仅选用单个核函数,在人脸特征提取方面仍显不足。鉴于此,提出多核Fisher判别分析法,即通过将多个单核Fisher判别得到的投影进行加权组合得到加权投影,以加权投影为依据进行特征提取和分类。实验表明,在进行人脸特征提取和分类时,多核Fisher判别分析法优于单核Fisher判别分析法。     

鉴别分析方法的最优化问题研究

提出了一种核Fisher鉴别分析方法优化方案,并分别给出了解决两类分类和解决多于两类的分类问题的算法,该方案具有明显的分类效率上的优势。在这种方案的实现中,首先从总体训练样本中选择出“显著”训练样本,对测试样本的分类只依赖于测试样本与“显著”训练样本之间的核函数。还设计出了一种选择“显著”训练样本的递归算法,以降低算法的计算复杂度。将该算法应用于人脸图象数据库与“基准”数据集,得到了很好的实验效果。     

Essence of kernel Fisher discriminant: KPCA plus LDA

In this paper, the method of kernel Fisher discriminant (KFD) is analyzed and its nature is revealed, i.e., KFD is equivalent to kernel principal component analysis (KPCA) plus Fisher linear discriminant analysis (LDA). Based on this result, a more transparent KFD algorithm is proposed. That is, KPCA is first performed and then LDA is used for a second feature extraction in the KPCA-transformed space. Finally, the effectiveness of the proposed algorithm is verified using the CENPARMI handwritten numeral database.     

基于散度差准则的隐空间特征抽取方法

本文提出了一种新的非线性特征抽取方法——基于散度差准则的隐空间特征抽取方法。该方法的主要思想就是首先利用一核函数将原始输入空间非线性变换到隐空间，然后，在该隐空间中，利用类间离散度与类内离散度之差作为鉴别准则进行特征抽取。与现有的核特征抽取方法不同，该方法不需要核函数满足Mercer定理，从而增加了核函数的选择范围。更为重要的是，由于采用了散度差作为鉴别准则，从根本上避免了传统的Fisher线性鉴别分析所遇到的小样本问题。在ORL人脸数据库和AR标准人脸库上的试验结果验证了本文方法的有效性。     

核典型相关性鉴别分析

提出一种新的基于典型相关性的核鉴别分析,以图片集为基础的人脸识别算法。把每个图片集映射到一个高维特征空间,然后通过核线性鉴别分析(KLDA)处理,得到相应的核子空间。通过计算两典型向量的典型差来估计两个子空间的相似度。根据核Fisher准则,基于类间典型差与类内典型差的比率建立核子空间的相关性来得到核典型相关性鉴别分析(KDCC)算法。在ORL、NUST603、FERNT和XM2VTS人脸库上的实验结果表明,该算法能够更有效提取样本特征,在识别率上要优于典型相关性鉴别分析(DCC)和核鉴别转换(KDT)算法。     

不相关鉴别分析改进算法

不相关鉴别分析是一种非常有效并起着重要作用的线性鉴别分析方法,它能抽取出具有不相关性质的特征分量。但是,由于每一个鉴别矢量的得出都要求解一个特征方程,不相关鉴别分析算法一直是计算代价很大的算法,在需求解的鉴别矢量个数较多时尤其如此。该文基于一个等效的Fisher准则函数,提出了不相关鉴别分析的另一问题模型。使用Lagrange乘子法,可求出对应该问题模型的“不相关”鉴别矢量解的简洁的表示式。关于CENPARMI手写体阿拉伯数字库和ORL人脸图象库的实验表明,该文提出的不相关鉴别分析改进算法计算效率较原算法有较大提高。     

基于核函数的Fisher判别分析算法在人耳识别中的应用

针对人耳图像自身的特点,并通过对现有生物识别技术的研究,本文尝试采用了一种基于核函数的Fisher判别分析算法对人耳进行识别。该算法不仅可以有效地提取人耳特征,获得较高的识别率;而且还可以解决因为光照和人耳旋转角度等因素带来的非线性问题。实验表明:采用基于径向基核函数的Fisher判别分析算法对人耳图像进行识别,其识别率最高,为98.701%。     

基于模糊核Fisher判别的说话人识别

在深入研究核Fisher判别方法的基础上,提出一种新的模糊核Fisher判别算法应用于说话人识别。采用模糊C均值聚类方法选择样本数据的同时,得到样本的模糊隶属度矩阵和聚类中心向量,进而对核Fisher判别算法中的类间离散度矩阵和类内离散度矩阵进行改进,生成模糊核Fisher判别算法,将其应用于说话人语音识别。     

基于支持向量的Kernel判别分析

提出了一种新的基于支持向苗的核化判别分析方法（SV—KFD）．首先深入地分析了支持向量机（SVM）以及核化费舍尔判别分析（Kernel Fisher）方法的相互关系．基于作者证明的SVM本身所同有的零空间性质;SVM分类面的法向量在基于支持向量的类内散度矩阵条件下,具有零空间特性,提山了利用SVM的法向量定义核化的决策边界特征矩阵（Kernelized Decision Boundary Feature Matrix,KDBFM）的方法．进一步结合均值向量的差向量构建扩展决策边界特征矩阵（Ex—KDBFM）．最后以支持向量为训练集合,结合零空间方法来计算投影空间,该投影空间被用来从原始图像中提取判别特征．以人脸识别为例,作者在FERET和CAS—PEAL—R1大规模人脸图像数据怍上对所提出的方法进行了实验验证,测试结果表明该方法具有比传统核判别分析方法更好的识别性能．     

On discovery of soft associations with “most” fuzzy quantifier for item promotion applications

In item promotion applications, there is a strong need for tools that can help to unlock the hidden profit within each individual customer’s transaction history. Discovering association patterns based on the data mining technique is helpful for this purpose. However, the conventional association mining approach, while generating “strong” association rules, cannot detect potential profit-building opportunities that can be exposed by “soft” association rules, which recommend items with looser but significant enough associations. This paper proposes a novel mining method that automatically detects hidden profit-building opportunities through discovering soft associations among items from historical transactions. Specifically, this paper proposes a relaxation method of association mining with a new support measurement, called soft support, that can be used for mining soft association patterns expressed with the “most” fuzzy quantifier. In addition, a novel measure for validating the soft-associated rules is proposed based on the estimated possibility of a conditioned quantified fuzzy event. The new measure is shown to be effective by comparison with several existing measures. A new association mining algorithm based on modification of the FT-Tree algorithm is proposed to accommodate this new support measure. Finally, the mining algorithm is applied to several data sets to investigate its effectiveness in finding soft patterns and content recommendation.