Kernel class-wise locality preserving projection

In the recent years, the pattern recognition community paid more attention to a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among them, locality preserving projection (LPP) is one of the most promising feature extraction techniques. However, when LPP is applied to the classification tasks, it shows some limitations, such as the ignorance of the label information. In this paper, we propose a novel local structure based feature extraction method, called class-wise locality preserving projection (CLPP). CLPP utilizes class information to guide the procedure of feature extraction. In CLPP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and the class information. The kernelized (nonlinear) counterpart of this linear feature extractor is also established in the paper. Moreover, a kernel version of CLPP namely Kernel CLPP (KCLPP) is developed through applying the kernel trick to CLPP to increase its performance on nonlinear feature extraction. Experiments on ORL face database and YALE face database are performed to test and evaluate the proposed algorithm.     

基于集成图的保局投影算法

为了克服保局投影方法(locality preserving projection,LPP)对噪音敏感,有效性依赖于近邻图构造等缺点,提出一种基于集成图的保局投影方法(graphs ensemble based LPP,GELPP).该方法先根据鲁棒统计原理定义出对噪声鲁棒的样本间相似性度量,再以该度量为基础构造多个近似的最大生成树;然后利用集成学习泛化能力强的优点来组合多个树为一个集成图;最后通过替换LPP的近邻图和相似性度量来进行保局投影.在高维人脸图像上的降维实验结果表明,该方法对噪声鲁棒,以及在集成图上降维的有效性.     

Optimal locality preserving projection for face recognition

In face recognition, when the number of images in the training set is much smaller than the number of pixels in each image, Locality Preserving Projections (LPP) often suffers from the singularity problem. To overcome singularity problem, principal component analysis is applied as a preprocessing step. But this procession may discard some important discriminative information. In this paper, a novel algorithm called Optimal Locality Preserving Projections (O-LPP) is proposed. The algorithm transforms the singular eigensystem computation to eigenvalue decomposition problems without losing any discriminative information, which can reduce the computation complexity. And the theoretical analysis related to the algorithm is also obtained. Extensive experiments on face databases demonstrate the proposed algorithm is superior to the traditional LPP algorithm.     

Discriminant sparse locality preserving projection for face recognition

Multimedia Tools and Applications - In this paper, aiming at the drawback of the popular dimensionality reduction method Discriminant Sparse Neighborhood Preserving Embedding(DSNPE), i.e. the...     

An adaptively weighted sub-pattern locality preserving projection for face recognition

In this paper, an adaptively weighted sub-pattern locality preserving projection (Aw-SpLPP) algorithm is proposed for face recognition. Unlike the traditional LPP algorithm which operates directly on the whole face image patterns and obtains a global face features that best detects the essential face manifold structure, the proposed Aw-SpLPP method operates on sub-patterns partitioned from an original whole face image and separately extracts corresponding local sub-features from them. Furthermore, the contribution of each sub-pattern can be adaptively computed by Aw-SpLPP in order to enhance the robustness to facial pose, expression and illumination variations. The efficiency of the proposed algorithm is demonstrated by extensive experiments on three standard face databases (Yale, YaleB and PIE). Experimental results show that Aw-SpLPP outperforms other holistic and sub-pattern based methods.     

Spatial-spectral locality preserving projection for hyperspectral image classification with limited training samples

Locality preserving projection (LPP) is a popular unsupervised feature extraction (FE) method. In this paper, the spatial-spectral LPP (SSLPP) method is proposed, which uses both the spectral and spatial information of hyperspectral image (HSI) for FE. The proposed method consists of two parts. In the first part, unlabelled samples are selected in a spatially homogeneous neighbourhood from filtered HSI. In the second part, the transformation matrix is calculated by an LPP-based method and by using the spectral and spatial information of the selected unlabelled samples. Experimental results on Indian Pines (IP), Kennedy Space Center (KSC), and Pavia University (PU) datasets show that the performance of SSLPP is superior to spectral unsupervised, supervised, and semi-supervised FE methods in small and large sample size situations. Moreover, the proposed method outperforms other spatial-spectral semi-supervised FE methods for PU dataset, which has high spatial resolution. For IP and KSC datasets, spectral regularized local discriminant embedding (SSRLDE) has the best performance by using spectral and spatial information of labelled and unlabelled samples, and SSLPP is ranked just behind it. Experiments show that SSLPP is an efficient unsupervised FE method, which does not use training samples as preparation of them is so difficult, costly, and sometimes impractical. SSLPP results are much better than LPP. Also, it decreases the storage and calculation costs using less number of unlabelled samples.     

Graph-optimized locality preserving projections

Locality preserving projections (LPP) is a typical graph-based dimensionality reduction (DR) method, and has been successfully applied in many practical problems such as face recognition. However, LPP depends mainly on its underlying neighborhood graph whose construction suffers from the following issues: (1) such neighborhood graph is artificially defined in advance, and thus does not necessary benefit subsequent DR task; (2) such graph is constructed using the nearest neighbor criterion which tends to work poorly due to the high-dimensionality of original space; (3) it is generally uneasy to assign appropriate values for the neighborhood size and heat kernel parameter involved in graph construction. To address these problems, we develop a novel DR algorithm called Graph-optimized Locality Preserving Projections (GoLPP). The idea is to integrate graph construction with specific DR process into a unified framework, which results in an optimized graph rather than predefined one. Moreover, an entropy regularization term is incorporated into the objective function for controlling the uniformity level of the edge weights in graph, so that a principled graph updating formula naturally corresponding to conventional heat kernel weights can be obtained. Finally, the experiments on several publicly available UCI and face data sets show the feasibility and effectiveness of the proposed method with encouraging results.     

RP-LPP : a random permutation based locality preserving projection for cancelable biometric recognition

Biometrics are being increasingly used across the world, but it also raises privacy and security concerns of the enrolled identities. The main reason is due to the fact that biometrics are not cancelable and if compromised may give access to the intruder. Cancelable biometric template is a solution to this problem which can be reissued if compromised. In this paper, we suggest a simple and powerful method called Random Permutation Locality Preserving Projection (RP-LPP) for Cancelable Biometric Recognition. Here, we exploit the mathematical relationship between the eigenvalues and eigenvectors of the original biometric image and its randomly permuted version is exploited for carrying out cancelable biometric recognition. The proposed technique work in a cryptic manner by accepting the cancelable biometric template and a key (called PIN) issued to a user. The effectiveness of the proposed techniques is demonstrated on three freely available face (ORL), iris (UBIRIS) and ear (IITD) datasets against state-of-the-art methods. The advantages of proposed technique are (i) the classification accuracy remains unaffected due to cancelable biometric templates generated using random permutation, (ii) security and quality of generated templates and (iii) robustness across different biometrics. In addition, no image registration is required for performing recognition.

     

A new linear discriminant analysis algorithm based on L1-norm maximization and locality preserving projection

Generic L2-norm-based linear discriminant analysis (LDA) is sensitive to outliers and only captures global structure information of sample points. In this paper, a new LDA-based feature extraction algorithm is proposed to integrate both global and local structure information via a unified L1-norm optimization framework. Unlike generic L2-norm-based LDA, the proposed algorithm explicitly incorporates the local structure information of sample points and is robust to outliers. It overcomes the problem of the singularity of within-class scatter matrix as well. Experiments on several popular datasets demonstrate the effectiveness of the proposed algorithm.     

Document clustering using locality preserving indexing

We propose a novel document clustering method which aims to cluster the documents into different semantic classes. The document space is generally of high dimensionality and clustering in such a high dimensional space is often infeasible due to the curse of dimensionality. By using locality preserving indexing (LPI), the documents can be projected into a lower-dimensional semantic space in which the documents related to the same semantics are close to each other. Different from previous document clustering methods based on latent semantic indexing (LSI) or nonnegative matrix factorization (NMF), our method tries to discover both the geometric and discriminating structures of the document space. Theoretical analysis of our method shows that LPI is an unsupervised approximation of the supervised linear discriminant analysis (LDA) method, which gives the intuitive motivation of our method. Extensive experimental evaluations are performed on the Reuters-21578 and TDT2 data sets.     

A locality correlation preserving support vector machine

This paper proposes a locality correlation preserving based support vector machine (LCPSVM) by combining the idea of margin maximization between classes and local correlation preservation of class data. It is a Support Vector Machine (SVM) like algorithm, which explicitly considers the locality correlation within each class in the margin and the penalty term of the optimization function. Canonical correlation analysis (CCA) is used to reveal the hidden correlations between two datasets, and a variant of correlation analysis model which implements locality preserving has been proposed by integrating local information into the objective function of CCA. Inspired by the idea used in canonical correlation analysis, we propose a locality correlation preserving within-class scatter matrix to replace the within-class scatter matrix in minimum class variance support machine (MCVSVM). This substitution has the property of keeping the locality correlation of data, and inherits the properties of SVM and other similar modified class of support vector machines. LCPSVM is discussed under linearly separable, small sample size and nonlinearly separable conditions, and experimental results on benchmark datasets demonstrate its effectiveness.     

Supervised projection approach for boosting classifiers

In this paper we present a new approach for boosting methods for the construction of ensembles of classifiers. The approach is based on using the distribution given by the weighting scheme of boosting to construct a non-linear supervised projection of the original variables, instead of using the weights of the instances to train the next classifier. With this method we construct ensembles that are able to achieve a better generalization error and are more robust to noise presence.It has been proved that AdaBoost method is able to improve the margin of the instances achieved by the ensemble. Moreover, its practical success has been partially explained by this margin maximization property. However, in noisy problems, likely to occur in real-world applications, the maximization of the margin of wrong instances or outliers can lead to poor generalization. We propose an alternative approach, where the distribution of the weights given by the boosting algorithm is used to get a supervised projection. Then, the supervised projection is used to train the next classifier using a uniform distribution of the training instances.The proposed approach is compared with three boosting techniques, namely AdaBoost, GentleBoost and MadaBoost, showing an improved performance on a large set of 55 problems from the UCI Machine Learning Repository, and less sensitiveness to noise in the class labels. The behavior of the proposed algorithm in terms of margin distribution and bias-variance decomposition is also studied.     

Regularized locality preserving discriminant embedding for face recognition

For face recognition, graph embedding techniques attempt to produce a high data locality projection for better recognition performance. However, estimation of population data locality could be severely biased due to small number of training samples. The biased estimation triggers overfitting problem and hence poor generalization. In this paper, we propose a new linear graph embedding technique based upon an adaptive locality preserving regulation model (ALPRM), known as Regularized Locality Preserving Discriminant Embedding (RLPDE). In RLPDE, the projection features are regulated based on ALPRM to approach population data locality, which can directly enhance the locality preserving capability of the projection features. This paper also presents the relation between locality preserving capability and class discrimination. Specifically, we show that the optimization of the locality preserving function minimizes the within-class variability. Experiments on three face datasets such as PIE, FRGC and FERET show the promising performance of the proposed technique.     

核的正交完备鉴别局部保持投影

针对完备鉴别局部保持投影算法所求得的最优判别矢量间存在信息冗余问题,提出了核的正交完备鉴别局部保持投影算法。通过将核函数技术与正交性原理融合,采用高斯核函数将原始样本映射到高维特征空间,在高维特征空间的局部总体散度矩阵中计算最优判别矢量,只需在整个范围内对值域空间进行特征值分解,去除局部零空间达到样本降维目的。该算法分别在 UMIST 人脸库和 JAFFE 人脸表情库上进行实验,实验结果表明算法的识别率高达95．59％。     

Learning a locality discriminating projection for classification

This paper introduces a new algorithm called locality discriminating projection (LDP) for subspace learning, which provides a new scheme for discriminant analysis by considering both the manifold structure and the prior class information. In the LDP algorithm, the overlap among the class-specific manifolds is approximated by an invader graph, and a locality discriminant criterion is proposed to find the projections that best preserve the within-class local structures while decrease the between-class overlap. The feasibility of the LDP algorithm has been successfully tested in text data and visual recognition experiments. Experiment results show it is an effective technique for data modeling and classification comparing to linear discriminant analysis, locality preserving projection, and marginal Fisher analysis.     

Regularized locality preserving discriminant analysis for face recognition

This paper proposes a regularized locality preserving discriminant analysis (RLPDA) approach for facial feature extraction and recognition. The RLPDA approach decomposes the eigenspace of the locality preserving within-class scatter matrix into three subspaces, i.e., the face space, the noise space and the null space, and then regularizes the three subspaces differently according to their predicted eigenvalues. As a result, the proposed approach integrates discriminative information in all of the three subspaces, de-emphasizes the effect of the eigenvectors corresponding to the small eigenvalues, and meanwhile suppresses the small sample size problem. Extensive experiments on ORL face database, FERET face subset and UMIST face database illustrate the effectiveness of the proposed approach.     

Exponential locality preserving projections for small sample size problem

Locality preserving projections (LPP) is a widely used manifold reduced dimensionality technique. However, it suffers from two problems: (1) small sample size problem and (2) the performance is sensitive to the neighborhood size k. In order to address these problems, we propose an exponential locality preserving projections (ELPP) by introducing the matrix exponential in this paper. ELPP avoids the singular of the matrices and obtains more valuable information for LPP. The experiments are conducted on three public face databases, ORL, Yale and Georgia Tech. The results show that the performances of ELPP is better than those of LPP and the state-of-the-art LPP Improved1.     

基于中心域的LPP算法研究

针对LPP算法中最近相邻图不能很好地表示流形的局部结构问题,提出一种基于中心域的保局投影算法。该算法采用LBP获取图像的高阶全局统计信息,并将其投影到LPP的流形空间。流形空间的获取是以各图像间的中心域的欧式距离为标准构建最近相邻图,使其可以简单地、较好地表示流形局部结构,并得到数量较少的特征维数。在ORL人脸库上的实验结果表明,该方法可有效地降低特征维数,并取得较好的识别率。     

Enhanced locality preserving projections using robust path based similarity

Curse of dimensionality is a bothering problem in high dimensional data analysis. To enhance the performances of classification or clustering on these data, their dimensionalities should be reduced beforehand. Locality Preserving Projections (LPP) is a widely used linear dimensionality reduction method. It seeks a subspace in which the neighborhood graph structure of samples is preserved. However, like most dimensionality reduction methods based on graph embedding, LPP is sensitive to noise and outliers, and its effectiveness depends on choosing suitable parameters for constructing the neighborhood graph. Unfortunately, it is difficult to choose effective parameters for LPP. To address these problems, we propose an Enhanced LPP (ELPP) using a similarity metric based on robust path and a Semi-supervised ELPP (SELPP) with pairwise constraints. In comparison with original LPP, our methods are not only robust to noise and outliers, but also less sensitive to parameters selection. Besides, SELPP makes use of pairwise constraints more efficiently than other comparing methods. Experimental results on real world face databases confirm their effectiveness.     

On minimum class locality preserving variance support vector machine

In this paper, a so-called minimum class locality preserving variance support machine (MCLPV_SVM) algorithm is presented by introducing the basic idea of the locality preserving projections (LPP), which can be seen as a modified class of support machine (SVM) and/or minimum class variance support machine (MCVSVM). MCLPV_SVM, in contrast to SVM and MCVSVM, takes the intrinsic manifold structure of the data space into full consideration and inherits the characteristics of SVM and MCVSVM. We discuss in the paper the linear case, the small sample size case and the nonlinear case of the MCLPV_SVM. Similar to MCVSVM, the MCLPV_SVM optimization problem in the small sample size case is solved by using dimensionality reduction through principal component analysis (PCA) and one in the nonlinear case is transformed into an equivalent linear MCLPV_SVM problem under kernel PCA (KPCA). Experimental results on real datasets indicate the effectiveness of the MCLPV_SVM by comparing it with SVM and MCVSVM.