Local sparse representation projections for face recognition

How to define the sparse affinity weight matrices is still an open problem in existing manifold learning algorithm. In this paper, we propose a novel supervised learning method called local sparse representation projections (LSRP) for linear dimensionality reduction. Differing from sparsity preserving projections (SPP) and the recent manifold learning methods such as locality preserving projections (LPP), LSRP introduces the local sparse representation information into the objective function. Although there are no labels used in the local sparse representation, it still can provide better measure coefficients and significant discriminant abilities. By combining the local interclass neighborhood relationships and sparse representation information, LSRP aims to preserve the local sparse reconstructive relationships of the data and simultaneously maximize the interclass separability. Comprehensive comparison and extensive experiments show that LSRP achieves higher recognition rates than principle component analysis, linear discriminant analysis and the state-of-the-art techniques such as LPP, SPP and maximum variance projections.     

Exponential locality preserving projections for small sample size problem

Locality preserving projections (LPP) is a widely used manifold reduced dimensionality technique. However, it suffers from two problems: (1) small sample size problem and (2) the performance is sensitive to the neighborhood size k. In order to address these problems, we propose an exponential locality preserving projections (ELPP) by introducing the matrix exponential in this paper. ELPP avoids the singular of the matrices and obtains more valuable information for LPP. The experiments are conducted on three public face databases, ORL, Yale and Georgia Tech. The results show that the performances of ELPP is better than those of LPP and the state-of-the-art LPP Improved1.     

Graph-optimized locality preserving projections

Locality preserving projections (LPP) is a typical graph-based dimensionality reduction (DR) method, and has been successfully applied in many practical problems such as face recognition. However, LPP depends mainly on its underlying neighborhood graph whose construction suffers from the following issues: (1) such neighborhood graph is artificially defined in advance, and thus does not necessary benefit subsequent DR task; (2) such graph is constructed using the nearest neighbor criterion which tends to work poorly due to the high-dimensionality of original space; (3) it is generally uneasy to assign appropriate values for the neighborhood size and heat kernel parameter involved in graph construction. To address these problems, we develop a novel DR algorithm called Graph-optimized Locality Preserving Projections (GoLPP). The idea is to integrate graph construction with specific DR process into a unified framework, which results in an optimized graph rather than predefined one. Moreover, an entropy regularization term is incorporated into the objective function for controlling the uniformity level of the edge weights in graph, so that a principled graph updating formula naturally corresponding to conventional heat kernel weights can be obtained. Finally, the experiments on several publicly available UCI and face data sets show the feasibility and effectiveness of the proposed method with encouraging results.     

Complete neighborhood preserving embedding for face recognition

Neighborhood preserving embedding (NPE) is a linear approximation to the locally linear embedding algorithm which can preserve the local neighborhood structure on the data manifold. However, in typical face recognition where the number of data samples is smaller than the dimension of data space, it is difficult to directly apply NPE to high dimensional matrices because of computational complexity. Moreover, in such case, NPE often suffers from the singularity problem of eigenmatrix, which makes the direct implementation of the NPE algorithm almost impossible. In practice, principal component analysis or singular value decomposition is applied as a preprocessing step to attack these problems. Nevertheless, this strategy may discard dimensions that contain important discriminative information and the eigensystem computation of NPE could be unstable. Towards a practical dimensionality reduction method for face data, we develop a new scheme in this paper, namely, the complete neighborhood preserving embedding (CNPE). CNPE transforms the singular generalized eigensystem computation of NPE into two eigenvalue decomposition problems. Moreover, a feasible and effective procedure is proposed to alleviate the computational burden of high dimensional matrix for typical face image data. Experimental results on the ORL face database and the Yale face database show that the proposed CNPE algorithm achieves better performance than other feature extraction methods, such as Eigenfaces, Fisherfaces and NPE, etc.     

中心近邻嵌入学习算法的人脸识别研究

针对人脸识别问题,提出了一种中心近邻嵌入的学习算法,其与经典的局部线性嵌入和保局映射不同,它是一种有监督的线性降维方法。该方法首先通过计算各类样本中心,并引入中心近邻距离代替两样本点之间的直接距离作为权系数函数的输入;然后再保持中心近邻的几何结构不变的情况下把高维数据嵌入到低维坐标系中。通过中心近邻嵌入学习算法与其他3种人脸识别方法(即主成分分析、线形判别分析及保局映射)在ORL、Yale及UMIST人脸库上进行的比较实验结果表明,它在高维数据低维可视化和人脸识别效果等方面均较其他3种方法取得了更好的效果。     

Face recognition using discriminative locality preserving vectors

We proposed an effective face recognition method based on the discriminative locality preserving vectors method (DLPV). Using the analysis of eigenspectrum modeling of locality preserving projections, we selected the reliable face variation subspace of LPP to construct the locality preserving vectors to characterize the data set. The discriminative locality preserving vectors (DLPV) method is based on the discriminant analysis on the locality preserving vectors. Furthermore, the theoretical analysis showed that the DLPV is viewed as a generalized discriminative common vector, null space linear discriminant analysis and null space discriminant locality preserving projections, which gave the intuitive motivation of our method. Extensive experimental results obtained on four well-known face databases (ORL, Yale, Extended Yale B and CMU PIE) demonstrated the effectiveness of the proposed DLPV method.     

无监督的差分鉴别特征提取以及在人脸识别上的应用

局部保持投影(LPP)只考虑了投影后的局部性,而忽视了非局部性.针对这个问题,引入非局部散布矩阵,提出无监督的差分鉴别特征提取算法,通过最大化非局部和局部之间的散度差来寻找最优变换矩阵,并将其成功地应用于人脸识别.该算法同时引入非局部和局部的信息,揭示隐含在高维图像空间中的非线性结构;采用差分的形式求解最优变换矩阵,以避免"小样本"问题;对LPP中的邻接矩阵进行了修正,以更准确地描述样本之间的邻近关系.在Yale和AR标准人脸库上的实验结果验证了文中算法的有效性.     

一种改进的邻域保持嵌入算法

邻域保持嵌入(NPE)是一种新颖的子空间学习算法,在降维的同时保持了样本集原有的局部邻域流形结构。为了进一步增强NPE在人脸识别和语音识别中的识别功能,提出了一种改进的邻域保持嵌入算法(RNPE)。在NPE的基础上通过引入类间权值矩阵,使得类间离散度最大,类内离散度最小,增加了样本类间散布约束。最后利用极端学习机(ELM)分类器进行分类,在Yale人脸库、Umist人脸库、Isolet语音库上的实验结果表明,RNPE算法的识别率明显高于NPE算法、LMMDE算法以及RAF-GE算法。     

基于双邻接图正交近邻保持投影的人脸识别算法

正交保持投影(ONPP)是经典的图嵌入降维技术,已经成功地应用到人脸识别中,其保持了高维数据的局部性和整体几何结构。监督的ONPP通过建立同类邻接图来最小化同类局部重构误差,寻找最优的低维嵌入,但是其只使用了类内信息,这会导致异类数据点间的结构不够明显。因此,提出了基于双邻接图的正交近邻保持投影(DAG-ONPP)算法。通过建立同类邻接图与异类邻接图,在数据嵌入低维空间后同类近邻重构误差尽量小,异类近邻重构误差更加明显。在ORL,Yale,YaleB和PIE人脸库上的实验结果表明,与其他经典算法相比,所提方法有效提高了分类能力。     

Nonlinear process monitoring based on kernel global–local preserving projections

A new nonlinear dimensionality reduction method called kernel global–local preserving projections (KGLPP) is developed and applied for fault detection. KGLPP has the advantage of preserving global and local data structures simultaneously. The kernel principal component analysis (KPCA), which only preserves the global Euclidean structure of data, and the kernel locality preserving projections (KLPP), which only preserves the local neighborhood structure of data, are unified in the KGLPP framework. KPCA and KLPP can be easily derived from KGLPP by choosing some particular values of parameters. As a result, KGLPP is more powerful than KPCA and KLPP in capturing useful data characteristics. A KGLPP-based monitoring method is proposed for nonlinear processes. T² and SPE statistics are constructed in the feature space for fault detection. Case studies in a nonlinear system and in the Tennessee Eastman process demonstrate that the KGLPP-based method significantly outperforms KPCA, KLPP and GLPP-based methods, in terms of higher fault detection rates and better fault sensitivity.     

基于二维近邻保持嵌入的图像识别

近邻保持嵌入算法(NPE)是一种保持数据流形上局部结构的子空间学习算法,它是对局部线性嵌入的线性逼近。然而当数据为图像时,图像被拉直为向量后的维数通常非常高,而样本点有限,由于矩阵的奇异性,NPE不能直接运用。我们将NPE推广到二维情形,提出二维近邻保持嵌入算法(2D-NPE)。2D-NPE直接在二维图像矩阵上提取图像特征,而不是把图像拉直成一维向量后再提取特征。通过在手写数字字符图像库和Yale人脸图像库上的实验,验证算法的有效性。     

基于鉴别稀疏保持嵌入的人脸识别算法

鉴于近年来稀疏表示（Sparse representation,SR）在高维数据例如人脸图像的特征提取与降维领域的快速发展,对原始的稀疏保持投影（Sparsity preserving projection,SPP）算法进行了改进,提出了一种叫做鉴别稀疏保持嵌入（Discriminant sparsity preserving embedding,DSPE）的算法. 通过求解一个最小二乘问题来更新SPP中的稀疏权重并得到一个更能真实反映鉴别信息的鉴别稀疏权重,最后以最优保持这个稀疏权重关系为目标来计算高维数据的低维特征子空间.该算法是一个线性的监督学习算法,通过引入鉴别信息,能够有效地对高维数据进行降维. 在ORL库、Yale库、扩展Yale B库和CMU PIE库上的大量实验结果验证了算法的有效性.     

基于大间距准则的不相关保局投影分析

局部保持投影(Locality preserving projections,LPP)算法只保持了目标在投影后的邻域局部信息,为了更好地刻画数据的流形结构, 引入了类内和类间局部散度矩阵,给出了一种基于有效且稳定的大间距准则(Maximum margin criterion,MMC)的不相关保局投影分析方法.该方法在最大化散度矩阵迹差时,引入尺度因子α,对类内和类间局部散度矩阵进行加权,以便找到更适合分类的子空间并且可避免小样本问题; 更重要的是,大间距准则下提取的判别特征集一般情况下是统计相关的,造成了特征信息的冗余, 因此,通过增加一个不相关约束条件,利用推导出的公式提取不相关判别特征集, 这样做, 对正确识别更为有利.在Yale人脸库、PIE人脸库和MNIST手写数字库上的测试结果表明,本文方法有效且稳定, 与LPP、LDA (Linear discriminant analysis)和LPMIP(Locality-preserved maximum information projection)方法等相比,具有更高的正确识别率.     

一种有监督的稀疏保持近邻嵌入算法

为充分利用样本的类别信息,提出一种有监督的稀疏保持近邻嵌入算法(SSNPE).该算法结合稀疏表示和保持近邻的思想,根据先验类标签信息保持局部邻域的固有几何关系.采用最小近邻分类器估算识别率,测试结果表明,在姿态、光照和表情变化的情况下,SSNPE都具有较高的识别率.     

Maximal local interclass embedding with application to face recognition

Dimensionality reduction of high dimensional data is involved in many problems in information processing. A new dimensionality reduction approach called maximal local interclass embedding (MLIE) is developed in this paper. MLIE can be viewed as a linear approach of a multimanifolds-based learning framework, in which the information of neighborhood is integrated with the local interclass relationships. In MLIE, the local interclass graph and the intrinsic graph are constructed to find a set of projections that maximize the local interclass scatter and the local intraclass compactness simultaneously. This characteristic makes MLIE more powerful than marginal Fisher analysis (MFA). MLIE maintains all the advantages of MFA. Moreover, the computational complexity of MLIE is less than that of MFA. The proposed algorithm is applied to face recognition. Experiments have been performed on the Yale, AR and ORL face image databases. The experimental results show that owing to the locally discriminating property, MLIE consistently outperforms up-to-date MFA, Smooth MFA, neighborhood preserving embedding and locality preserving projection in face recognition.     

Two-dimensional complete neighborhood preserving embedding

Complete neighborhood preserving embedding (CNPE) is an improvement to the neighborhood preserving embedding (NPE) algorithm, which can address the singularity and stability problems of NPE and at the same time preserve useful discriminative information. However, CNPE works with vectorized representations of data, and thus, the original 2D face image matrices should be previously transformed into the same dimensional vectors. Such a matrix-to-vector transform usually leads to a high-dimensional image vector space, which makes the eigenanalysis quite difficult and time-consuming. Beyond computational issues, some spatial structural information between nearby pixels may be lost after vectorization. In this paper, we develop a new scheme for image feature extraction, namely, two-dimensional complete neighborhood preserving embedding (2D-CNPE). 2D-CNPE builds the eigenmatrix and the weight matrix which characterize local neighborhood properties of data directly based on the original face images, and then, the optimal embedding axes are obtained by performing an eigen-decomposition. Experimental results on three face databases show that the proposed 2D-CNPE achieves better performance than other feature extraction methods, such as Eigenfaces, Fisherfaces, and 2D-PCA.     

Globally maximizing, locally minimizing: unsupervised discriminant projection with applications to face and palm biometrics

This paper develops an unsupervised discriminant projection (UDP) technique for dimensionality reduction of high-dimensional data in small sample size cases. UDP can be seen as a linear approximation of a multimanifolds-based learning framework which takes into account both the local and nonlocal quantities. UDP characterizes the local scatter as well as the nonlocal scatter, seeking to find a projection that simultaneously maximizes the nonlocal scatter and minimizes the local scatter. This characteristic makes UDP more intuitive and more powerful than the most up-to-date method, locality preserving projection (LPP), which considers only the local scatter for clustering or classification tasks. The proposed method is applied to face and palm biometrics and is examined using the Yale, FERET, and AR face image databases and the PolyU palmprint database. The experimental results show that UDP consistently outperforms LPP and PCA and outperforms LDA when the training sample size per class is small. This demonstrates that UDP is a good choice for real-world biometrics applications     

Kernel class-wise locality preserving projection

In the recent years, the pattern recognition community paid more attention to a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among them, locality preserving projection (LPP) is one of the most promising feature extraction techniques. However, when LPP is applied to the classification tasks, it shows some limitations, such as the ignorance of the label information. In this paper, we propose a novel local structure based feature extraction method, called class-wise locality preserving projection (CLPP). CLPP utilizes class information to guide the procedure of feature extraction. In CLPP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and the class information. The kernelized (nonlinear) counterpart of this linear feature extractor is also established in the paper. Moreover, a kernel version of CLPP namely Kernel CLPP (KCLPP) is developed through applying the kernel trick to CLPP to increase its performance on nonlinear feature extraction. Experiments on ORL face database and YALE face database are performed to test and evaluate the proposed algorithm.     

A structure-preserved local matching approach for face recognition

In this paper, a novel local matching method called structure-preserved projections (SPP) is proposed for face recognition. Unlike most existing local matching methods which neglect the interactions of different sub-pattern sets during feature extraction, i.e., they assume different sub-pattern sets are independent; SPP takes the holistic context of the face into account and can preserve the configural structure of each face image in subspace. Moreover, the intrinsic manifold structure of the sub-pattern sets can also be preserved in our method. With SPP, all sub-patterns partitioned from the original face images are trained to obtain a unified subspace, in which recognition can be performed. The efficiency of the proposed algorithm is demonstrated by extensive experiments on three standard face databases (Yale, Extended YaleB and PIE). Experimental results show that SPP outperforms other holistic and local matching methods.     

Sparse two-dimensional local discriminant projections for feature extraction

Two-dimensional local graph embedding discriminant analysis (2DLGEDA) and two-dimensional discriminant locality preserving projections (2DDLPP) were recently proposed to directly extract features form 2D face matrices to improve the performance of two-dimensional locality preserving projections (2DLPP). But all of them require a high computational cost and the learned transform matrices lack intuitive and semantic interpretations. In this paper, we propose a novel method called sparse two-dimensional locality discriminant projections (S2DLDP), which is a sparse extension of graph-based image feature extraction method. S2DLDP combines the spectral analysis and L₁-norm regression using the Elastic Net to learn the sparse projections. Differing from the existing 2D methods such as 2DLPP, 2DDLP and 2DLGEDA, S2DLDP can learn the sparse 2D face profile subspaces (also called sparsefaces), which give an intuitive, semantic and interpretable feature subspace for face representation. We point out that using S2DLDP for face feature extraction is, in essence, to project the 2D face images on the semantic face profile subspaces, on which face recognition is also performed. Experiments on Yale, ORL and AR face databases show the efficiency and effectiveness of S2DLDP.