利用标准化LDA进行人脸识别

线性判别分析（LDA）是一种较为普遍的用于特征提取的线性分类方法。提出一种基于LDA的人脸识别方法－－标准化LDA，该方法克服了传统LDA方法的缺点，重新定义了样本类间离散度矩阵，在原始定义的基础上增加一个由类间距离决定的可变权函数，使得在选择投地，能够更好地分开各个类的样本；同时，它采用一种合理而有效的方法解决矩阵奇异的问题，即保留样本类内离散度矩阵的零空间，因为这个空间包含了最具有判别能力的信息。在这个零空间里，寻找对应于样本类间离散度矩阵的较大特征值的特征向量作为最后降维的转换矩阵。实验结果显示，在人脸识别中，与传统LDA相比，该方法有更好的识别率。标准化LDA也可以用于其他图像识别问题。     

一种改进的线性判别分析算法在人脸识别中的应用

线性判别分析算法是一种经典的特征提取方法,但其仅在大样本情况下适用。本文针对传统线性判别分析算法面临的小样本问题和秩限制问题,提出了一种改进的线性判别分析算法ILDA。该方法在矩阵指数的基础上,重新定义了类内离散度矩阵和类间离散度矩阵,有效地同时提取类内离散度矩阵零空间和非零空间中的信息。若干人脸数据库上的比较实验表明了ILDA在人脸识别方面的有效性。     

Kernel-based improved discriminant analysis and its application to face recognition

Kernel discriminant analysis (KDA) is a widely used tool in feature extraction community. However, for high-dimensional multi-class tasks such as face recognition, traditional KDA algorithms have the limitation that the Fisher criterion is nonoptimal with respect to classification rate. Moreover, they suffer from the small sample size problem. This paper presents a variant of KDA called kernel-based improved discriminant analysis (KIDA), which can effectively deal with the above two problems. In the proposed framework, origin samples are projected firstly into a feature space by an implicit nonlinear mapping. After reconstructing between-class scatter matrix in the feature space by weighted schemes, the kernel method is used to obtain a modified Fisher criterion directly related to classification error. Finally, simultaneous diagonalization technique is employed to find lower-dimensional nonlinear features with significant discriminant power. Experiments on face recognition task show that the proposed method is superior to the traditional KDA and LDA.     

Facial Feature Extraction Method Based on Coefficients of Variances

Principal Component Analysis （PCA） and Linear Discriminant Analysis （LDA） are two popular feature extraction techniques in statistical pattern recognition field. Due to small sample size problem LDA cannot be directly applied to appearance-based face recognition tasks. As a consequence, a lot of LDA-based facial feature extraction techniques are proposed to deal with the problem one after the other. Nullspace Method is one of the most effective methods among them. The Nullspace Method tries to find a set of discriminant vectors which maximize the between-class scatter in the null space of the within-class scatter matrix. The calculation of its discriminant vectors will involve performing singular value decomposition on a high-dimensional matrix. It is generally memory- and time-consuming. Borrowing the key idea in Nullspace method and the concept of coefficient of variance in statistical analysis we present a novel facial feature extraction method, i.e., Discriminant based on Coefficient of Variance （DCV） in this paper. Experimental results performed on the FERET and AR face image databases demonstrate that DCV is a promising technique in comparison with Eigenfaces, Nullspace Method, and other state-of-the-art facial feature extraction methods.     

改进的线性判别分析算法

线性判别分析是一种有效的特征提取方法,但其存在两个缺陷：小样本问题和秩限制问题。为了解决上述问题,提出一种改进的线性判别分析算法ILDA。该方法引进类间离散度标量和类内离散度标量,通过求解样本各维的权值达到特征提取的目的。若干标准人脸数据集和人工数据集上的实验表明ILDA在特征提取方面的有效性。     

Feature Extraction Using Laplacian Maximum Margin Criterion

Feature extraction by Maximum Margin Criterion (MMC) can more efficiently calculate the discriminant vectors than LDA, by avoiding calculation of the inverse within-class scatter matrix. But MMC ignores the local structures of samples. In this paper, we develop a novel criterion to address this issue, namely Laplacian Maximum Margin Criterion (Laplacian MMC). We define the total Laplacian matrix, within-class Laplacian matrix and between-class Laplacian matrix by using the similar weight of samples to capture the scatter information. Laplacian MMC based feature extraction gets the discriminant vectors by maximizing the difference between between-class laplacian matrix and within-class laplacian matrix. Experiments on FERET and AR face databases show that Laplacian MMC works well.     

基于大间距准则的不相关保局投影分析

局部保持投影(Locality preserving projections,LPP)算法只保持了目标在投影后的邻域局部信息,为了更好地刻画数据的流形结构, 引入了类内和类间局部散度矩阵,给出了一种基于有效且稳定的大间距准则(Maximum margin criterion,MMC)的不相关保局投影分析方法.该方法在最大化散度矩阵迹差时,引入尺度因子α,对类内和类间局部散度矩阵进行加权,以便找到更适合分类的子空间并且可避免小样本问题; 更重要的是,大间距准则下提取的判别特征集一般情况下是统计相关的,造成了特征信息的冗余, 因此,通过增加一个不相关约束条件,利用推导出的公式提取不相关判别特征集, 这样做, 对正确识别更为有利.在Yale人脸库、PIE人脸库和MNIST手写数字库上的测试结果表明,本文方法有效且稳定, 与LPP、LDA (Linear discriminant analysis)和LPMIP(Locality-preserved maximum information projection)方法等相比,具有更高的正确识别率.     

Feature extraction based on Laplacian bidirectional maximum margin criterion

Maximum margin criterion (MMC) based feature extraction is more efficient than linear discriminant analysis (LDA) for calculating the discriminant vectors since it does not need to calculate the inverse within-class scatter matrix. However, MMC ignores the discriminative information within the local structures of samples and the structural information embedding in the images. In this paper, we develop a novel criterion, namely Laplacian bidirectional maximum margin criterion (LBMMC), to address the issue. We formulate the image total Laplacian matrix, image within-class Laplacian matrix and image between-class Laplacian matrix using the sample similar weight that is widely used in machine learning. The proposed LBMMC based feature extraction computes the discriminant vectors by maximizing the difference between image between-class Laplacian matrix and image within-class Laplacian matrix in both row and column directions. Experiments on the FERET and Yale face databases show the effectiveness of the proposed LBMMC based feature extraction method.     

A multi-manifold discriminant analysis method for image feature extraction

In this paper, we propose a Multi-Manifold Discriminant Analysis (MMDA) method for an image feature extraction and pattern recognition based on graph embedded learning and under the Fisher discriminant analysis framework. In an MMDA, the within-class graph and between-class graph are, respectively, designed to characterize the within-class compactness and the between-class separability, seeking for the discriminant matrix to simultaneously maximize the between-class scatter and minimize the within-class scatter. In addition, in an MMDA, the within-class graph can represent the sub-manifold information, while the between-class graph can represent the multi-manifold information. The proposed MMDA is extensively examined by using the FERET, AR and ORL face databases, and the PolyU finger-knuckle-print databases. The experimental results demonstrate that an MMDA is effective in feature extraction, leading to promising image recognition performance.     

类依赖的线性判别分析

线性判别分析(LDA)是一种常用的特征提取方法,其目标是提取特征后样本的类间离散度和类内离散度的比值最大,即各类样本在特征空间中有最佳的可分离性.该方法利用同一个准则将所有类的样本投影到同一个特征空间中,忽略了各类样本分布特征的差异.本文提出类依赖的线性判别方法(Class-Specific LDA,CSLDA),对每一类样本寻找最优的投影矩阵,使得投影后能够更好地把该类样本与所有其他类的样本尽可能分开,并将该方法与经验核相结合,得到经验核空间中类依赖的线性判别分析.在人工数据集和UCI数据集上的实验结果表明,在输入空间和经验核空间里均有CSLDA特征提取后的识别率高于LDA.     

A Gradient Linear Discriminant Analysis for Small Sample Sized Problem

The purpose of conventional linear discriminant analysis (LDA) is to find an orientation which projects high dimensional feature vectors of different classes to a more manageable low dimensional space in the most discriminative way for classification. The LDA technique utilizes an eigenvalue decomposition (EVD) method to find such an orientation. This computation is usually adversely affected by the small sample size problem. In this paper we have presented a new direct LDA method (called gradient LDA) for computing the orientation especially for small sample size problem. The gradient descent based method is used for this purpose. It also avoids discarding the null space of within-class scatter matrix and between-class scatter matrix which may have discriminative information useful for classification.     

样本典型性分析与线性鉴别分析

首先分析了经典LDA方法的物理意义及其局限性,然后提出了一个新的LDA方法。该方法强调训练样本的典型性与代表性,并认为相同类别中与一个样本距离较远的若干样本是同一类别中对这个样本有典型意义的样本,而不同类别中与这个样本距离较近的若干样本也是对该样本而言有典型代表意义的样本。该新的LDA方法基于定义在这些典型样本上的类间散布矩阵与类内散布矩阵实现特征提取。方法的物理意义体现为:特征提取过程中最大化样本与不同类中的典型样本间距离与最小化样本与同类中的典型样本间距离这一思路的实现,可使抽取出的不同类别的样本特征具有更大的线性可分离性。充分的理论与实验分析表明本文方法可优于经典LDA方法。     

A multiple maximum scatter difference discriminant criterion for facial feature extraction.

Maximum scatter difference (MSD) discriminant criterion was a recently presented binary discriminant criterion for pattern classification that utilizes the generalized scatter difference rather than the generalized Rayleigh quotient as a class separability measure, thereby avoiding the singularity problem when addressing small-sample-size problems. MSD classifiers based on this criterion have been quite effective on face-recognition tasks, but as they are binary classifiers, they are not as efficient on large-scale classification tasks. To address the problem, this paper generalizes the classification-oriented binary criterion to its multiple counterpart--multiple MSD (MMSD) discriminant criterion for facial feature extraction. The MMSD feature-extraction method, which is based on this novel discriminant criterion, is a new subspace-based feature-extraction method. Unlike most other subspace-based feature-extraction methods, the MMSD computes its discriminant vectors from both the range of the between-class scatter matrix and the null space of the within-class scatter matrix. The MMSD is theoretically elegant and easy to calculate. Extensive experimental studies conducted on the benchmark database, FERET, show that the MMSD out-performs state-of-the-art facial feature-extraction methods such as null space method, direct linear discriminant analysis (LDA), eigenface, Fisherface, and complete LDA.     

An optimal symmetrical null space criterion of Fisher discriminant for feature extraction and recognition

Linear discriminant analysis (LDA) is one of the most effective feature extraction methods in statistical pattern recognition, which extracts the discriminant features by maximizing the so-called Fisher’s criterion that is defined as the ratio of between-class scatter matrix to within-class scatter matrix. However, classification of high-dimensional statistical data is usually not amenable to standard pattern recognition techniques because of an underlying small sample size (SSS) problem. A popular approach to the SSS problem is the removal of non-informative features via subspace-based decomposition techniques. Motivated by this viewpoint, many elaborate subspace decomposition methods including Fisherface, direct LDA (D-LDA), complete PCA plus LDA (C-LDA), random discriminant analysis (RDA) and multilinear discriminant analysis (MDA), etc., have been developed, especially in the context of face recognition. Nevertheless, how to search a set of complete optimal subspaces for discriminant analysis is still a hot topic of research in area of LDA. In this paper, we propose a novel discriminant criterion, called optimal symmetrical null space (OSNS) criterion that can be used to compute the Fisher’s maximal discriminant criterion combined with the minimal one. Meanwhile, by the reformed criterion, the complete symmetrical subspaces based on the within-class and between-class scatter matrices are constructed, respectively. Different from the traditional subspace learning criterion that derives only one principal subspace, in our approach two null subspaces and their orthogonal complements were all obtained through the optimization of OSNS criterion. Therefore, the algorithm based on OSNS has the potential to outperform the traditional LDA algorithms, especially in the cases of small sample size. Experimental results conducted on the ORL, FERET, XM2VTS and NUST603 face image databases demonstrate the effectiveness of the proposed method.     

基于类间散布矩阵的二维主分量分析

主分量分析是一种线性特征抽取方法,被广泛地应用在人脸等图像识别领域。但传统的PCA都以总体散布矩阵作为产生矩阵,并且要将作为图像的矩阵转换为列向量进行计算。该文给出了一种利用图像矩阵直接计算的二维PCA,以类间散布矩阵的本征向量作为投影方向,取得了比利用总体散布矩阵更好的识别效果,并且特征抽取速度更快。在ORL和NUSTFDBⅡ标准人脸库上的实验验证了该方法的有效性。     

Feature extraction using fuzzy inverse FDA

This paper proposes a new method of feature extraction and recognition, namely, the fuzzy inverse Fisher discriminant analysis (FIFDA) based on the inverse Fisher discriminant criterion and fuzzy set theory. In the proposed method, a membership degree matrix is calculated using FKNN, then the membership degree is incorporated into the definition of the between-class scatter matrix and within-class scatter matrix to get the fuzzy between-class scatter matrix and fuzzy within-class scatter matrix. Experimental results on the ORL, FERET face databases and pulse signal database show that the new method outperforms Fisherface, fuzzy Fisherface and inverse Fisher discriminant analysis.     

一种新的图像特征抽取方法研究

对最佳鉴别矢量的求解方法进行了研究，根据矩阵的分块理论和优化理论，在一定的条件下，从理论上得到类间散布矩阵和总体散布矩阵的一种简洁表示方法，提出了求解最佳鉴别矢量的一种新算法，该算法的优点是计算量明显减少。ORL人脸数据库的数值实验，验证了上述论断的正确性。实验结果表明，虽然识别率与分块维数之间存在非线性关系，但可以通过选择适当的分块维数来获得较高的识别率。类间散布矩阵和总体散布矩阵的一种简洁表示方法适合于一切使用Fisher鉴别准则的模式识别问题。     

Kernel machine-based one-parameter regularized fisher discriminant method for face recognition.

This paper addresses two problems in linear discriminant analysis (LDA) of face recognition. The first one is the problem of recognition of human faces under pose and illumination variations. It is well known that the distribution of face images with different pose, illumination, and face expression is complex and nonlinear. The traditional linear methods, such as LDA, will not give a satisfactory performance. The second problem is the small sample size (S3) problem. This problem occurs when the number of training samples is smaller than the dimensionality of feature vector. In turn, the within-class scatter matrix will become singular. To overcome these limitations, this paper proposes a new kernel machine-based one-parameter regularized Fisher discriminant (K1PRFD) technique. K1PRFD is developed based on our previously developed one-parameter regularized discriminant analysis method and the well-known kernel approach. Therefore, K1PRFD consists of two parameters, namely the regularization parameter and kernel parameter. This paper further proposes a new method to determine the optimal kernel parameter in RBF kernel and regularized parameter in within-class scatter matrix simultaneously based on the conjugate gradient method. Three databases, namely FERET, Yale Group B, and CMU PIE, are selected for evaluation. The results are encouraging. Comparing with the existing LDA-based methods, the proposed method gives superior results.     

一种改进的线性判别分析法在人脸识别中的应用

提出了一种新的基于LDA的人脸识别算法。该方法重新定义了样本的类间散布矩阵，在原始的定义基础上增加了一种径向基函数（RBF）调节类间距离，使得在选择投影方向时能更好地分升各类样本；同时该方法存类间散布矩阵与类内散布矩阵的特征分解的基础上，通过变换求出符合Fisher准则的最优投影方向，可以证明这样得到的投影方向同时具有正交性与统计不相关性。通过ORL人脸数据库的数值实验，表明了该算法比传统算法有更好的性能。     

A new hybrid feature extraction method in a dyadic scheme for classification of hyperspectral data

The feature extraction is an important preprocessing step of the classification procedure particularly in high-dimensional data with limited number of training samples. Conventional supervised feature extraction methods, for example, linear discriminant analysis (LDA), generalized discriminant analysis, and non-parametric weighted feature extraction ones, need to calculate scatter matrices. In these methods, within-class and between-class scatter matrices are used to formulate the criterion of class separability. Because of the limited number of training samples, the accurate estimation of these matrices is not possible. So the classification accuracy of these methods falls in a small sample size situation. To cope with this problem, a new supervised feature extraction method namely, feature extraction using attraction points (FEUAP) has been recently proposed in which no statistical moments are used. Thus, it works well using limited training samples. To take advantage of this method and LDA one, this article combines them by a dyadic scheme. In the proposed scheme, the similar classes are grouped hierarchically by the k-means algorithm so that a tree with some nodes is constructed. Then the class of each pixel is determined from this scheme. To determine the class of each pixel, depending on the node of the tree, we use FEUAP or LDA for a limited or large number of training samples, respectively. The experimental results demonstrate the better performance of the proposed hybrid method in comparison with other supervised feature extraction methods in a small sample size situation.