Kernel machine-based one-parameter regularized fisher discriminant method for face recognition.

This paper addresses two problems in linear discriminant analysis (LDA) of face recognition. The first one is the problem of recognition of human faces under pose and illumination variations. It is well known that the distribution of face images with different pose, illumination, and face expression is complex and nonlinear. The traditional linear methods, such as LDA, will not give a satisfactory performance. The second problem is the small sample size (S3) problem. This problem occurs when the number of training samples is smaller than the dimensionality of feature vector. In turn, the within-class scatter matrix will become singular. To overcome these limitations, this paper proposes a new kernel machine-based one-parameter regularized Fisher discriminant (K1PRFD) technique. K1PRFD is developed based on our previously developed one-parameter regularized discriminant analysis method and the well-known kernel approach. Therefore, K1PRFD consists of two parameters, namely the regularization parameter and kernel parameter. This paper further proposes a new method to determine the optimal kernel parameter in RBF kernel and regularized parameter in within-class scatter matrix simultaneously based on the conjugate gradient method. Three databases, namely FERET, Yale Group B, and CMU PIE, are selected for evaluation. The results are encouraging. Comparing with the existing LDA-based methods, the proposed method gives superior results.     

Pose Robust Low-resolution Face Recognition via Coupled Kernel-based Enhanced Discriminant Analysis

Most face recognition techniques have been successful in dealing with high-resolution (HR) frontal face images. However, real-world face recognition systems are often confronted with the low-resolution (LR) face images with pose and illumination variations. This is a very challenging issue, especially under the constraint of using only a single gallery image per person. To address the problem, we propose a novel approach called coupled kernel-based enhanced discriminant analysis (CKEDA). CKEDA aims to simultaneously project the features from LR non-frontal probe images and HR frontal gallery ones into a common space where discrimination property is maximized. There are four advantages of the proposed approach: 1) by using the appropriate kernel function, the data becomes linearly separable, which is beneficial for recognition; 2) inspired by linear discriminant analysis (LDA), we integrate multiple discriminant factors into our objective function to enhance the discrimination property; 3) we use the gallery extended trick to improve the recognition performance for a single gallery image per person problem; 4) our approach can address the problem of matching LR non-frontal probe images with HR frontal gallery images, which is difficult for most existing face recognition techniques. Experimental evaluation on the multi-PIE dataset signifies highly competitive performance of our algorithm.      

A comparison of generalized linear discriminant analysis algorithms

Linear discriminant analysis (LDA) is a dimension reduction method which finds an optimal linear transformation that maximizes the class separability. However, in undersampled problems where the number of data samples is smaller than the dimension of data space, it is difficult to apply LDA due to the singularity of scatter matrices caused by high dimensionality. In order to make LDA applicable, several generalizations of LDA have been proposed recently. In this paper, we present theoretical and algorithmic relationships among several generalized LDA algorithms and compare their computational complexities and performances in text classification and face recognition. Towards a practical dimension reduction method for high dimensional data, an efficient algorithm is proposed, which reduces the computational complexity greatly while achieving competitive prediction accuracies. We also present nonlinear extensions of these LDA algorithms based on kernel methods. It is shown that a generalized eigenvalue problem can be formulated in the kernel-based feature space, and generalized LDA algorithms are applied to solve the generalized eigenvalue problem, resulting in nonlinear discriminant analysis. Performances of these linear and nonlinear discriminant analysis algorithms are compared extensively.     

The kernel common vector method: a novel nonlinear subspace classifier for pattern recognition.

The common vector (CV) method is a linear subspace classifier method which allows one to discriminate between classes of data sets, such as those arising in image and word recognition. This method utilizes subspaces that represent classes during classification. Each subspace is modeled such that common features of all samples in the corresponding class are extracted. To accomplish this goal, the method eliminates features that are in the direction of the eigenvectors corresponding to the nonzero eigenvalues of the covariance matrix of each class. In this paper, we introduce a variation of the CV method, which will be referred to as the modified CV (MCV) method. Then, a novel approach is proposed to apply the MCV method in a nonlinearly mapped higher dimensional feature space. In this approach, all samples are mapped into a higher dimensional feature space using a kernel mapping function, and then, the MCV method is applied in the mapped space. Under certain conditions, each class gives rise to a unique CV, and the method guarantees a 100% recognition rate with respect to the training set data. Moreover, experiments with several test cases also show that the generalization performance of the proposed kernel method is comparable to the generalization performances of other linear subspace classifier methods as well as the kernel-based nonlinear subspace method. While both the MCV method and its kernel counterpart did not outperform the support vector machine (SVM) classifier in most of the reported experiments, the application of our proposed methods is simpler than that of the multiclass SVM classifier. In addition, it is not necessary to adjust any parameters in our approach.     

Kernel-based nonlinear discriminant analysis for face recognition

Linear subspace analysis methods have been successfully applied to extract features for face recognition.But they are inadequate to represent the complex and nonlinear variations of real face images,such as illumination,facial expression and pose variations,because of their linear properties.In this paper,a nonlinear subspace analysis method,Kernel-based Nonlinear Discriminant Analysis (KNDA),is presented for face recognition,which combines the nonlinear kernel trick with the linear subspace analysis method-Fisher Linear Discriminant Analysis (FLDA).First,the kernel trick is used to project the input data into an implicit feature space,then FLDA is performed in this feature space.Thus nonlinear discriminant features of the input data are yielded.In addition,in order to reduce the computational complexity,a geometry-based feature vectors selection scheme is adopted.Another similar nonlinear subspace analysis is Kernel-based Principal Component Analysis (KPCA),which combines the kernel trick with linear Principal Component Analysis (PCA).Experiments are performed with the polynomial kernel,and KNDA is compared with KPCA and FLDA.Extensive experimental results show that KNDA can give a higher recognition rate than KPCA and FLDA.     

人脸识别中基于核的子空间鉴别分析

尽管基于Fisher准则的线性鉴别分析被公认为特征抽取的有效方法之一，并被成功地用于人脸识别，但是由于光照变化、人脸表情和姿势变化，实际上的人脸图像分布是十分复杂的，因此，抽取非线性鉴别特征显得十分必要。为了能利用非线性鉴别特征进行人脸识别，提出了一种基于核的子空间鉴别分析方法。该方法首先利用核函数技术将原始样本隐式地映射到高维（甚至无穷维）特征空间；然后在高维特征空间里，利用再生核理论来建立基于广义Fisher准则的两个等价模型；最后利用正交补空间方法求得最优鉴别矢量来进行人脸识别。在ORL和NUST603两个人脸数据库上，对该方法进行了鉴别性能实验，得到了识别率分别为94％和99．58％的实验结果，这表明该方法与核组合方法的识别结果相当，且明显优于KPCA和Kernel fisherfaces方法的识别结果。     

An optimal symmetrical null space criterion of Fisher discriminant for feature extraction and recognition

Linear discriminant analysis (LDA) is one of the most effective feature extraction methods in statistical pattern recognition, which extracts the discriminant features by maximizing the so-called Fisher’s criterion that is defined as the ratio of between-class scatter matrix to within-class scatter matrix. However, classification of high-dimensional statistical data is usually not amenable to standard pattern recognition techniques because of an underlying small sample size (SSS) problem. A popular approach to the SSS problem is the removal of non-informative features via subspace-based decomposition techniques. Motivated by this viewpoint, many elaborate subspace decomposition methods including Fisherface, direct LDA (D-LDA), complete PCA plus LDA (C-LDA), random discriminant analysis (RDA) and multilinear discriminant analysis (MDA), etc., have been developed, especially in the context of face recognition. Nevertheless, how to search a set of complete optimal subspaces for discriminant analysis is still a hot topic of research in area of LDA. In this paper, we propose a novel discriminant criterion, called optimal symmetrical null space (OSNS) criterion that can be used to compute the Fisher’s maximal discriminant criterion combined with the minimal one. Meanwhile, by the reformed criterion, the complete symmetrical subspaces based on the within-class and between-class scatter matrices are constructed, respectively. Different from the traditional subspace learning criterion that derives only one principal subspace, in our approach two null subspaces and their orthogonal complements were all obtained through the optimization of OSNS criterion. Therefore, the algorithm based on OSNS has the potential to outperform the traditional LDA algorithms, especially in the cases of small sample size. Experimental results conducted on the ORL, FERET, XM2VTS and NUST603 face image databases demonstrate the effectiveness of the proposed method.     

Kernel quadratic discriminant analysis for small sample size problem

It is generally believed that quadratic discriminant analysis (QDA) can better fit the data in practical pattern recognition applications compared to linear discriminant analysis (LDA) method. This is due to the fact that QDA relaxes the assumption made by LDA-based methods that the covariance matrix for each class is identical. However, it still assumes that the class conditional distribution is Gaussian which is usually not the case in many real-world applications. In this paper, a novel kernel-based QDA method is proposed to further relax the Gaussian assumption by using the kernel machine technique. The proposed method solves the complex pattern recognition problem by combining the QDA solution and the kernel machine technique, and at the same time, tackles the so-called small sample size problem through a regularized estimation of the covariance matrix. Extensive experimental results indicate that the proposed method is a more sophisticated solution outperforming many traditional kernel-based learning algorithms.     

Kernel machine-based rank-lifting regularized discriminant analysis method for face recognition

To address two problems, namely nonlinear problem and singularity problem, of linear discriminant analysis (LDA) approach in face recognition, this paper proposes a novel kernel machine-based rank-lifting regularized discriminant analysis (KRLRDA) method. A rank-lifting theorem is first proven using linear algebraic theory. Combining the rank-lifting strategy with three-to-one regularization technique, the complete regularized methodology is developed on the within-class scatter matrix. The proposed regularized scheme not only adjusts the projection directions but tunes their corresponding weights as well. Moreover, it is shown that the final regularized within-class scatter matrix approaches to the original one as the regularized parameter tends to zero. Two public available databases, namely FERET and CMU PIE face databases, are selected for evaluations. Compared with some existing kernel-based LDA methods, the proposed KRLRDA approach gives superior performance.     

Face recognition using direct, weighted linear discriminant analysis and modular subspaces

We present a modular linear discriminant analysis (LDA) approach for face recognition. A set of observers is trained independently on different regions of frontal faces and each observer projects face images to a lower-dimensional subspace. These lower-dimensional subspaces are computed using LDA methods, including a new algorithm that we refer to as direct, weighted LDA or DW-LDA. DW-LDA combines the advantages of two recent LDA enhancements, namely direct LDA (D-LDA) and weighted pairwise Fisher criteria. Each observer performs recognition independently and the results are combined using a simple sum-rule. Experiments compare the proposed approach to other face recognition methods that employ linear dimensionality reduction. These experiments demonstrate that the modular LDA method performs significantly better than other linear subspace methods. The results also show that D-LDA does not necessarily perform better than the well-known principal component analysis followed by LDA approach. This is an important and significant counterpoint to previously published experiments that used smaller databases. Our experiments also indicate that the new DW-LDA algorithm is an improvement over D-LDA.     

基于规范化KDDA的人脸识别算法

传统的PCA和LDA算法受限于“小样本问题”,且对像素的高阶相关性不敏感。论文将核函数方法与规范化LDA相结合,将原图像空间通过非线性映射变换到高维特征空间,并借助于“核技巧”在新的空间中应用鉴别分析方法。通过对ORL人脸库的大量实验表明,该方法在特征提取方面优于PCA,KPCA,LDA等其他方法,在简化分类器的同时,也可以获得高识别率。     

Face recognition using laplacianfaces

We propose an appearance-based face recognition method called the Laplacianface approach. By using locality preserving projections (LPP), the face images are mapped into a face subspace for analysis. Different from principal component analysis (PCA) and linear discriminant analysis (LDA) which effectively see only the Euclidean structure of face space, LPP finds an embedding that preserves local information, and obtains a face subspace that best detects the essential face manifold structure. The Laplacianfaces are the optimal linear approximations to the eigenfunctions of the Laplace Beltrami operator on the face manifold. In this way, the unwanted variations resulting from changes in lighting, facial expression, and pose may be eliminated or reduced. Theoretical analysis shows that PCA, LDA, and LPP can be obtained from different graph models. We compare the proposed Laplacianface approach with Eigenface and Fisherface methods on three different face data sets. Experimental results suggest that the proposed Laplacianface approach provides a better representation and achieves lower error rates in face recognition.     

一种基于核的快速非线性鉴别分析方法

基于“核技巧”提出的新的非线性鉴别分析方法在最小二乘意义上与基于核的Fisher鉴别分析方法等效,相应鉴别方向通过一个线性方程组得出,计算代价较小,相应分类实现极其简便．该方法的最大优点是,对训练数据进行筛选,可使构造鉴别矢量的“显著”训练模式数大大低于总训练模式数,从而使得测试集的分类非常高效;同时,设计出专门的优化算法以加速“显著”训练模式的选取．实验表明,这种非线性方法不仅具有明显的效率上的优势,且具有不低于基于核的Fisher鉴别分析方法的性能．     

Modified locally linear discriminant embedding for plant leaf recognition

Based on locally linear embedding (LLE) and modified maximizing margin criterion (MMMC), a modified locally linear discriminant embedding (MLLDE) algorithm is proposed for plant leaf recognition in this paper. By MLLDE, the plant leaf images are mapped into a leaf subspace for analysis, which can detect the essential leaf manifold structure. Furthermore, the unwanted variations resulting from changes in period, location, and illumination can be eliminated or reduced. Different from principal component analysis (PCA) and linear discriminant analysis (LDA), which can only deal with flat Euclidean structures of plant leaf space, MLLDE not only inherits the advantages of locally linear embedding (LLE), but makes full use of class information to improve discriminant power by introducing translation and rescaling models. The experimental results on real plant leaf database show that the MLLDE is effective for plant leaf recognition.     

基于规范化KDDA的人脸识别

传统的PCA和LDA算法受限于“小样本问题”,且对象素的高阶相关性不敏感。本文将核函数方法与规范化LDA相结合,将原图像空间通过非线性映射变换到高维特征空间,并借助于“核技巧”在新的空间中应用鉴别分析方法。通过对ORL人脸库的大量实验研究表明,本文方法在特征提取方面明显优于PCA,KPCA,LDA等其他传统的人脸识别方法,在简化分类器的同时,也可以获得高识别率。     

Learning the kernel matrix by maximizing a KFD-based class separability criterion

The advantage of a kernel method often depends critically on a proper choice of the kernel function. A promising approach is to learn the kernel from data automatically. In this paper, we propose a novel method for learning the kernel matrix based on maximizing a class separability criterion that is similar to those used by linear discriminant analysis (LDA) and kernel Fisher discriminant (KFD). It is interesting to note that optimizing this criterion function does not require inverting the possibly singular within-class scatter matrix which is a computational problem encountered by many LDA and KFD methods. We have conducted experiments on both synthetic data and real-world data from UCI and FERET, showing that our method consistently outperforms some previous kernel learning methods.     

Face recognition using Gabor-based direct linear discriminant analysis and support vector machine

This paper presents a novel and uniform framework for face recognition. This framework is based on a combination of Gabor wavelets, direct linear discriminant analysis (DLDA) and support vector machine (SVM). First, feature vectors are extracted from raw face images using Gabor wavelets. These Gabor-based features are robust against local distortions caused by the variance of illumination, expression and pose. Next, the extracted feature vectors are projected to a low-dimensional subspace using DLDA technique. The Gabor-based DLDA feature vectors are then applied to SVM classifier. A new kernel function for SVM called hyperhemispherically normalized polynomial (HNP) is also proposed in this paper and its validity on the improvement of classification accuracy is theoretically proved and experimentally tested for face recognition. The proposed algorithm was evaluated using the FERET database. Experimental results show that the proposed face recognition system outperforms other related approaches in terms of recognition rate.     

Robust large margin discriminant tangent analysis for face recognition

Fisher’s Linear Discriminant Analysis (LDA) has been recognized as a powerful technique for face recognition. However, it could be stranded in the non-Gaussian case. Nonparametric discriminant analysis (NDA) is a typical algorithm that extends LDA from Gaussian case to non-Gaussian case. However, NDA suffers from outliers and unbalance problems, which cause a biased estimation of the extra-class scatter information. To address these two problems, we propose a robust large margin discriminant tangent analysis method. A tangent subspace-based algorithm is first proposed to learn a subspace from a set of intra-class and extra-class samples which are distributed in a balanced way on the local manifold patch near each sample point, so that samples from the same class are clustered as close as possible and samples from different classes will be separated far away from the tangent center. Then each subspace is aligned to a global coordinate by tangent alignment. Finally, an outlier detection technique is further proposed to learn a more accurate decision boundary. Extensive experiments on challenging face recognition data set demonstrate the effectiveness and efficiency of the proposed method for face recognition. Compared to other nonparametric methods, the proposed one is more robust to outliers.     

融合判别式深度特征学习的图像识别算法

目的 卷积神经网络在图像识别算法中得到了广泛应用。针对传统卷积神经网络学习到的特征缺少更有效的鉴别能力而导致图像识别性能不佳等问题,提出一种融合线性判别式思想的损失函数LDloss（linear discriminant loss）并用于图像识别中的深度特征提取,以提高特征的鉴别能力,进而改善图像识别性能。方法 首先利用卷积神经网络搭建特征提取所需的深度网络,然后在考虑样本分类误差最小化的基础上,对于图像多分类问题,引入LDA（linear discriminant analysis）思想构建新的损失函数参与卷积神经网络的训练,来最小化类内特征距离和最大化类间特征距离,以提高特征的鉴别能力,从而进一步提高图像识别性能,分析表明,本文算法可以获得更有助于样本分类的特征。其中,学习过程中采用均值分批迭代更新的策略实现样本均值平稳更新。结果 该算法在MNIST数据集和CK+数据库上分别取得了99.53%和94.73%的平均识别率,与现有算法相比较有一定的提升。同时,与传统的损失函数Softmax loss和Hinge loss对比,采用LDloss的深度网络在MNIST数据集上分别提升了0.2%和0.3%,在CK+数据库上分别提升了9.21%和24.28%。结论 本文提出一种新的融合判别式深度特征学习算法,该算法能有效地提高深度网络的可鉴别能力,从而提高图像识别精度,并且在测试阶段,与Softmax loss相比也不需要额外的计算量。     

Visual learning and recognition of 3-d objects from appearance

The problem of automatically learning object models for recognition and pose estimation is addressed. In contrast to the traditional approach, the recognition problem is formulated as one of matching appearance rather than shape. The appearance of an object in a two-dimensional image depends on its shape, reflectance properties, pose in the scene, and the illumination conditions. While shape and reflectance are intrinsic properties and constant for a rigid object, pose and illumination vary from scene to scene. A compact representation of object appearance is proposed that is parametrized by pose and illumination. For each object of interest, a large set of images is obtained by automatically varying pose and illumination. This image set is compressed to obtain a low-dimensional subspace, called the eigenspace, in which the object is represented as a manifold. Given an unknown input image, the recognition system projects the image to eigenspace. The object is recognized based on the manifold it lies on. The exact position of the projection on the manifold determines the object's pose in the image.A variety of experiments are conducted using objects with complex appearance characteristics. The performance of the recognition and pose estimation algorithms is studied using over a thousand input images of sample objects. Sensitivity of recognition to the number of eigenspace dimensions and the number of learning samples is analyzed. For the objects used, appearance representation in eigenspaces with less than 20 dimensions produces accurate recognition results with an average pose estimation error of about 1.0 degree. A near real-time recognition system with 20 complex objects in the database has been developed. The paper is concluded with a discussion on various issues related to the proposed learning and recognition methodology.