Face recognition using discriminant locality preserving projections based on maximum margin criterion

In this paper, we propose a new discriminant locality preserving projections based on maximum margin criterion (DLPP/MMC). DLPP/MMC seeks to maximize the difference, rather than the ratio, between the locality preserving between-class scatter and locality preserving within-class scatter. DLPP/MMC is theoretically elegant and can derive its discriminant vectors from both the range of the locality preserving between-class scatter and the range space of locality preserving within-class scatter. DLPP/MMC can also derive its discriminant vectors from the null space of locality preserving within-class scatter when the parameter of DLPP/MMC approaches +∞. Experiments on the ORL, Yale, FERET, and PIE face databases show the effectiveness of the proposed DLPP/MMC.     

Graph-optimized locality preserving projections

Locality preserving projections (LPP) is a typical graph-based dimensionality reduction (DR) method, and has been successfully applied in many practical problems such as face recognition. However, LPP depends mainly on its underlying neighborhood graph whose construction suffers from the following issues: (1) such neighborhood graph is artificially defined in advance, and thus does not necessary benefit subsequent DR task; (2) such graph is constructed using the nearest neighbor criterion which tends to work poorly due to the high-dimensionality of original space; (3) it is generally uneasy to assign appropriate values for the neighborhood size and heat kernel parameter involved in graph construction. To address these problems, we develop a novel DR algorithm called Graph-optimized Locality Preserving Projections (GoLPP). The idea is to integrate graph construction with specific DR process into a unified framework, which results in an optimized graph rather than predefined one. Moreover, an entropy regularization term is incorporated into the objective function for controlling the uniformity level of the edge weights in graph, so that a principled graph updating formula naturally corresponding to conventional heat kernel weights can be obtained. Finally, the experiments on several publicly available UCI and face data sets show the feasibility and effectiveness of the proposed method with encouraging results.     

Bagging null space locality preserving discriminant classifiers for face recognition

In this paper, we propose a novel bagging null space locality preserving discriminant analysis (bagNLPDA) method for facial feature extraction and recognition. The bagNLPDA method first projects all the training samples into the range space of a so-called locality preserving total scatter matrix without losing any discriminative information. The projected training samples are then randomly sampled using bagging to generate a set of bootstrap replicates. Null space discriminant analysis is performed in each replicate and the results of them are combined using majority voting. As a result, the proposed method aggregates a set of complementary null space locality preserving discriminant classifiers. Experiments on FERET and PIE subsets demonstrate the effectiveness of bagNLPDA.     

Facial expression recognition based on two-dimensional discriminant locality preserving projections

In this paper, a novel method called two-dimensional discriminant locality preserving projections (2D-DLPP) is proposed. By introducing between-class scatter constraint and label information into two-dimensional locality preserving projections (2D-LPP) algorithm, 2D-DLPP successfully finds the subspace which can best discriminate different pattern classes. So the subspace obtained by 2D-DLPP has more discriminant power than 2D-LPP, and is more suitable for recognition tasks. The proposed method was applied to facial expression recognition tasks on JAFFE and Cohn-Kanade database and compared with other three widely used two-dimensional methods: 2D-PCA, 2D-LDA and 2D-LPP. The high recognition rates show the effectiveness of the proposed algorithm.     

Enhanced locality preserving projections using robust path based similarity

Curse of dimensionality is a bothering problem in high dimensional data analysis. To enhance the performances of classification or clustering on these data, their dimensionalities should be reduced beforehand. Locality Preserving Projections (LPP) is a widely used linear dimensionality reduction method. It seeks a subspace in which the neighborhood graph structure of samples is preserved. However, like most dimensionality reduction methods based on graph embedding, LPP is sensitive to noise and outliers, and its effectiveness depends on choosing suitable parameters for constructing the neighborhood graph. Unfortunately, it is difficult to choose effective parameters for LPP. To address these problems, we propose an Enhanced LPP (ELPP) using a similarity metric based on robust path and a Semi-supervised ELPP (SELPP) with pairwise constraints. In comparison with original LPP, our methods are not only robust to noise and outliers, but also less sensitive to parameters selection. Besides, SELPP makes use of pairwise constraints more efficiently than other comparing methods. Experimental results on real world face databases confirm their effectiveness.     

核的正交完备鉴别局部保持投影

针对完备鉴别局部保持投影算法所求得的最优判别矢量间存在信息冗余问题,提出了核的正交完备鉴别局部保持投影算法。通过将核函数技术与正交性原理融合,采用高斯核函数将原始样本映射到高维特征空间,在高维特征空间的局部总体散度矩阵中计算最优判别矢量,只需在整个范围内对值域空间进行特征值分解,去除局部零空间达到样本降维目的。该算法分别在 UMIST 人脸库和 JAFFE 人脸表情库上进行实验,实验结果表明算法的识别率高达95．59％。     

Regularized locality preserving discriminant analysis for face recognition

This paper proposes a regularized locality preserving discriminant analysis (RLPDA) approach for facial feature extraction and recognition. The RLPDA approach decomposes the eigenspace of the locality preserving within-class scatter matrix into three subspaces, i.e., the face space, the noise space and the null space, and then regularizes the three subspaces differently according to their predicted eigenvalues. As a result, the proposed approach integrates discriminative information in all of the three subspaces, de-emphasizes the effect of the eigenvectors corresponding to the small eigenvalues, and meanwhile suppresses the small sample size problem. Extensive experiments on ORL face database, FERET face subset and UMIST face database illustrate the effectiveness of the proposed approach.     

Two-dimensional locality preserving projections (2DLPP) with its application to palmprint recognition

This paper proposes a novel algorithm for image feature extraction, namely, the two-dimensional locality preserving projections (2DLPP), which directly extracts the proper features from image matrices based on locality preserving criterion. Experimental results on the PolyU palmprint database show the effectiveness of the proposed algorithm.     

Improved discriminant locality preserving projections for face and palmprint recognition

We propose in this paper two improved manifold learning methods called diagonal discriminant locality preserving projections (Dia-DLPP) and weighted two-dimensional discriminant locality preserving projections (W2D-DLPP) for face and palmprint recognition. Motivated by the fact that diagonal images outperform the original images for conventional two-dimensional (2D) subspace learning methods such as 2D principal component analysis (2DPCA) and 2D linear discriminant analysis (2DLDA), we first propose applying diagonal images to a recently proposed 2D discriminant locality preserving projections (2D-DLPP) algorithm, and formulate the Dia-DLPP method for feature extraction of face and palmprint images. Moreover, we show that transforming an image to a diagonal image is equivalent to assigning an appropriate weight to each pixel of the original image to emphasize its different importance for recognition, which provides the rationale and superiority of using diagonal images for 2D subspace learning. Inspired by this finding, we further propose a new discriminant weighted method to explicitly calculate the discriminative score of each pixel within a face and palmprint sample to duly emphasize its different importance, and incorporate it into 2D-DLPP to formulate the W2D-DLPP method to improve the recognition performance of 2D-DLPP and Dia-DLPP. Experimental results on the widely used FERET face and PolyU palmprint databases demonstrate the efficacy of the proposed methods.     

Inverse Fisher discriminate criteria for small sample size problem and its application to face recognition

This paper addresses the small sample size problem in linear discriminant analysis, which occurs in face recognition applications. Belhumeur et al. [IEEE Trans. Pattern Anal. Mach. Intell. 19 (7) (1997) 711-720] proposed the FisherFace method. We find out that the FisherFace method might fail since after the PCA transform the corresponding within class covariance matrix can still be singular, this phenomenon is verified with the Yale face database. Hence we propose to use an inverse Fisher criteria. Our method works when the number of training images per class is one. Experiment results suggest that this new approach performs well.     

Multivariate statistical process control based on multiway locality preserving projections

An approach for multivariate statistical process control based on multiway locality preserving projections (LPP) is presented. The recently developed LPP is a linear dimensionality reduction technique for preserving the neighborhood structure of the data set. It is characterized by capturing the intrinsic structure of the observed data and finding more meaningful low-dimensional information hidden in the high-dimensional observations compared with PCA. In this study, LPP is used to extract the intrinsic geometrical structure of the process data. Hotelling’s T² (D) and the squared prediction error (SPE or Q) statistic charts for on-line monitoring are then presented, and the contribution plots of these two statistical indices are used for fault diagnosis. Moreover, a moving window technique is used for the implementation of on-line monitoring. Case study was carried out with the data of industrial penicillin fed-batch cultivations. As a comparison, the results obtained with the MPCA are also presented. It is concluded that the Multiway LPP (MLPP) outperforms the conventional MPCA. Finally, the robustness of the MLPP monitoring is analyzed by adding noises to the original data.     

Regularized discriminant analysis for the small sample size problem in face recognition

It is well-known that the applicability of both linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA) to high-dimensional pattern classification tasks such as face recognition (FR) often suffers from the so-called “small sample size” (SSS) problem arising from the small number of available training samples compared to the dimensionality of the sample space. In this paper, we propose a new QDA like method that effectively addresses the SSS problem using a regularization technique. Extensive experimentation performed on the FERET database indicates that the proposed methodology outperforms traditional methods such as Eigenfaces, direct QDA and direct LDA in a number of SSS setting scenarios.     

Regularized locality preserving discriminant embedding for face recognition

For face recognition, graph embedding techniques attempt to produce a high data locality projection for better recognition performance. However, estimation of population data locality could be severely biased due to small number of training samples. The biased estimation triggers overfitting problem and hence poor generalization. In this paper, we propose a new linear graph embedding technique based upon an adaptive locality preserving regulation model (ALPRM), known as Regularized Locality Preserving Discriminant Embedding (RLPDE). In RLPDE, the projection features are regulated based on ALPRM to approach population data locality, which can directly enhance the locality preserving capability of the projection features. This paper also presents the relation between locality preserving capability and class discrimination. Specifically, we show that the optimization of the locality preserving function minimizes the within-class variability. Experiments on three face datasets such as PIE, FRGC and FERET show the promising performance of the proposed technique.     

On minimum class locality preserving variance support vector machine

In this paper, a so-called minimum class locality preserving variance support machine (MCLPV_SVM) algorithm is presented by introducing the basic idea of the locality preserving projections (LPP), which can be seen as a modified class of support machine (SVM) and/or minimum class variance support machine (MCVSVM). MCLPV_SVM, in contrast to SVM and MCVSVM, takes the intrinsic manifold structure of the data space into full consideration and inherits the characteristics of SVM and MCVSVM. We discuss in the paper the linear case, the small sample size case and the nonlinear case of the MCLPV_SVM. Similar to MCVSVM, the MCLPV_SVM optimization problem in the small sample size case is solved by using dimensionality reduction through principal component analysis (PCA) and one in the nonlinear case is transformed into an equivalent linear MCLPV_SVM problem under kernel PCA (KPCA). Experimental results on real datasets indicate the effectiveness of the MCLPV_SVM by comparing it with SVM and MCVSVM.     

Generalized nonlinear discriminant analysis and its small sample size problems

This paper develops a generalized nonlinear discriminant analysis (GNDA) method and deals with its small sample size (SSS) problems. GNDA is a nonlinear extension of linear discriminant analysis (LDA), while kernel Fisher discriminant analysis (KFDA) can be regarded as a special case of GNDA. In LDA, an under sample problem or a small sample size problem occurs when the sample size is less than the sample dimensionality, which will result in the singularity of the within-class scatter matrix. Due to a high-dimensional nonlinear mapping in GNDA, small sample size problems arise rather frequently. To tackle this issue, this research presents five different schemes for GNDA to solve the SSS problems. Experimental results on real-world data sets show that these schemes for GNDA are very effective in tackling small sample size problems.     

Improving implementation of linear discriminant analysis for the high dimension/small sample size problem

Classification based on Fisher's linear discriminant analysis (FLDA) is challenging when the number of variables largely exceeds the number of given samples. The original FLDA needs to be carefully modified and with high dimensionality implementation issues like reduction of storage costs are of crucial importance. Methods are reviewed for the high dimension/small sample size problem and the one closest, in some sense, to the classical regular approach is chosen. The implementation of this method with regard to computational and storage costs and numerical stability is improved. This is achieved through combining a variety of known and new implementation strategies. Experiments demonstrate the superiority, with respect to both overall costs and classification rates, of the resulting algorithm compared with other methods.     

LPP solution schemes for use with face recognition

Locality preserving projection (LPP) is a manifold learning method widely used in pattern recognition and computer vision. The face recognition application of LPP is known to suffer from a number of problems including the small sample size (SSS) problem, the fact that it might produce statistically identical transform results for neighboring samples, and that its classification performance seems to be heavily influenced by its parameters. In this paper, we propose three novel solution schemes for LPP. Experimental results also show that the proposed LPP solution scheme is able to classify much more accurately than conventional LPP and to obtain a classification performance that is only little influenced by the definition of neighbor samples.     

Optimal locality preserving projection for face recognition

In face recognition, when the number of images in the training set is much smaller than the number of pixels in each image, Locality Preserving Projections (LPP) often suffers from the singularity problem. To overcome singularity problem, principal component analysis is applied as a preprocessing step. But this procession may discard some important discriminative information. In this paper, a novel algorithm called Optimal Locality Preserving Projections (O-LPP) is proposed. The algorithm transforms the singular eigensystem computation to eigenvalue decomposition problems without losing any discriminative information, which can reduce the computation complexity. And the theoretical analysis related to the algorithm is also obtained. Extensive experiments on face databases demonstrate the proposed algorithm is superior to the traditional LPP algorithm.     

A generalised K-L expansion method which can deal with small sample size and high-dimensional problems

The K-L expansion method, which is able to extract the discriminatory information contained in class-mean vectors, is generalised, in this paper, to make it suitable for solving small sample size problems. We further investigate, theoretically, how to reduce the method’s computational complexity in high-dimensional cases. As a result, a simple and efficient GKLE algorithm is developed. We test our method on the ORL face image database and the NUST603 handwritten Chinese character database, and our experimental results demonstrate that GKLE outperforms the existing techniques of PCA, PCA plus LDA, and Direct LDA. ID="A1"Correspondance and offprint requests to: J-Y. Yang, Department of Computer Science, Nanjing University of Science and Technology, Nanjiung 210094, P.R. China. E-mail: csiyang@comp.poly.edu.hk     

广义的鉴别局部中值保持投影及人脸识别

针对鉴别的局部中值保持投影(DLMPP)在小样本情况下面临的类内散布矩阵奇异的问题,提出了广义的鉴别局部中值保持投影(GDLMPP)算法。GDLMPP首先将样本等价映射到一个低维子空间,然后在此子空间求解最佳投影矩阵,从而有效解决了小样本问题,并从理论上验证了当类内散布矩阵非奇异时,GDLMPP等价于DLMPP。最后,通过在ORL及AR库上的实验验证了算法的有效性。