最坏分离的联合分辨率判别分析

现实中,常需辨识低分辨率(low-resolution,简称LR)图像(如监控系统所捕捉的人脸),但相比通常的高(high-resolution,简称HR)或超(super-resolution,简称SR)分辨率图像而言,其含有相对较少的判别信息,致使通常的子空间学习算法,如结合主成分分析(principal components analysis,简称PCA)的线性判别分析(linear discriminant analysis,简称LDA)难以获得理想的识别效果.为了缓和该问题,最近所提出的联合判别分析(如SDA)借助与低分辨率相配对的高分辨率图像辅助设计LR图像分类器.在SDA的实现中,其采用了类似LDA的平均散度定义,使SDA遗传了LDA在投影时难以使相对靠近的类充分分离的问题.为了克服该不足,提出了针对LR图像识别的最坏分离的联合分辨率判别分析(worst-separated couple-resolution discriminant analysis,简称WSCR),从而使:(1) LR和HR投影到同一低维子空间;(2) 投影后的最小类间隔最大化.实验结果表明:与SDA相比,WSCR更适用于低分辨率的图像识别.     

近邻类鉴别分析方法

提出一种近邻类鉴别分析方法,线性鉴别分析是该方法的一个特例。线性鉴别分析通过最大化类间散度同时最小化类内散度寻找最佳投影,其中类间散度是所有类之间散度的总体平均;而近邻类鉴别分析中类间散度定义为各个类与其k个近邻类之间的平均散度。该方法通过选取适当的近邻类数,能够缓解线性鉴别降维后造成的部分类的重叠。实验结果表明近邻类鉴别分析方法性能稳定且优于传统的线性鉴别分析。     

基于大间距准则的不相关保局投影分析

局部保持投影(Locality preserving projections,LPP)算法只保持了目标在投影后的邻域局部信息,为了更好地刻画数据的流形结构, 引入了类内和类间局部散度矩阵,给出了一种基于有效且稳定的大间距准则(Maximum margin criterion,MMC)的不相关保局投影分析方法.该方法在最大化散度矩阵迹差时,引入尺度因子α,对类内和类间局部散度矩阵进行加权,以便找到更适合分类的子空间并且可避免小样本问题; 更重要的是,大间距准则下提取的判别特征集一般情况下是统计相关的,造成了特征信息的冗余, 因此,通过增加一个不相关约束条件,利用推导出的公式提取不相关判别特征集, 这样做, 对正确识别更为有利.在Yale人脸库、PIE人脸库和MNIST手写数字库上的测试结果表明,本文方法有效且稳定, 与LPP、LDA (Linear discriminant analysis)和LPMIP(Locality-preserved maximum information projection)方法等相比,具有更高的正确识别率.     

An optimal symmetrical null space criterion of Fisher discriminant for feature extraction and recognition

Linear discriminant analysis (LDA) is one of the most effective feature extraction methods in statistical pattern recognition, which extracts the discriminant features by maximizing the so-called Fisher’s criterion that is defined as the ratio of between-class scatter matrix to within-class scatter matrix. However, classification of high-dimensional statistical data is usually not amenable to standard pattern recognition techniques because of an underlying small sample size (SSS) problem. A popular approach to the SSS problem is the removal of non-informative features via subspace-based decomposition techniques. Motivated by this viewpoint, many elaborate subspace decomposition methods including Fisherface, direct LDA (D-LDA), complete PCA plus LDA (C-LDA), random discriminant analysis (RDA) and multilinear discriminant analysis (MDA), etc., have been developed, especially in the context of face recognition. Nevertheless, how to search a set of complete optimal subspaces for discriminant analysis is still a hot topic of research in area of LDA. In this paper, we propose a novel discriminant criterion, called optimal symmetrical null space (OSNS) criterion that can be used to compute the Fisher’s maximal discriminant criterion combined with the minimal one. Meanwhile, by the reformed criterion, the complete symmetrical subspaces based on the within-class and between-class scatter matrices are constructed, respectively. Different from the traditional subspace learning criterion that derives only one principal subspace, in our approach two null subspaces and their orthogonal complements were all obtained through the optimization of OSNS criterion. Therefore, the algorithm based on OSNS has the potential to outperform the traditional LDA algorithms, especially in the cases of small sample size. Experimental results conducted on the ORL, FERET, XM2VTS and NUST603 face image databases demonstrate the effectiveness of the proposed method.     

Face recognition using discriminant locality preserving projections based on maximum margin criterion

In this paper, we propose a new discriminant locality preserving projections based on maximum margin criterion (DLPP/MMC). DLPP/MMC seeks to maximize the difference, rather than the ratio, between the locality preserving between-class scatter and locality preserving within-class scatter. DLPP/MMC is theoretically elegant and can derive its discriminant vectors from both the range of the locality preserving between-class scatter and the range space of locality preserving within-class scatter. DLPP/MMC can also derive its discriminant vectors from the null space of locality preserving within-class scatter when the parameter of DLPP/MMC approaches +∞. Experiments on the ORL, Yale, FERET, and PIE face databases show the effectiveness of the proposed DLPP/MMC.     

Feature Extraction Using Laplacian Maximum Margin Criterion

Feature extraction by Maximum Margin Criterion (MMC) can more efficiently calculate the discriminant vectors than LDA, by avoiding calculation of the inverse within-class scatter matrix. But MMC ignores the local structures of samples. In this paper, we develop a novel criterion to address this issue, namely Laplacian Maximum Margin Criterion (Laplacian MMC). We define the total Laplacian matrix, within-class Laplacian matrix and between-class Laplacian matrix by using the similar weight of samples to capture the scatter information. Laplacian MMC based feature extraction gets the discriminant vectors by maximizing the difference between between-class laplacian matrix and within-class laplacian matrix. Experiments on FERET and AR face databases show that Laplacian MMC works well.     

中心线邻域鉴别嵌入算法及其在人脸识别中的应用*

针对边界费舍尔分析在特征提取过程中存在的不足,提出中心线邻域鉴别嵌入(CLNDE)算法,并应用于人脸识别中.CLNDE首先利用样本到类中心线的距离分别构造类内相似矩阵与类间相似矩阵;然后利用构造的相似矩阵计算样本的类间局部散度与类内局部散度;最后在最大化样本的类间局部散度的同时最小化类内局部散度,寻求最优投影矩阵.在人脸数据库上实验验证算法的优越性.     

新的局部保持鉴别分析算法

提出了一种新的局部保持鉴别分析算法：基于迹比准则与自适应近邻图嵌入的局部保持鉴别分析算法。根据样本分布特性自适应构建类内和类间近邻图,保持数据的局部结构并且利用数据的鉴别信息,定义局部类内离差矩阵以及局部类间离差矩阵,采用迹比Fisher判别函数作为目标函数,通过迭代的方法最大化局部类间离差矩阵与类内离差矩阵的迹比值,解得最优子空间。在ORL和Yale人脸数据库上的实验表明该方法是有效的。     

Feature extraction based on Laplacian bidirectional maximum margin criterion

Maximum margin criterion (MMC) based feature extraction is more efficient than linear discriminant analysis (LDA) for calculating the discriminant vectors since it does not need to calculate the inverse within-class scatter matrix. However, MMC ignores the discriminative information within the local structures of samples and the structural information embedding in the images. In this paper, we develop a novel criterion, namely Laplacian bidirectional maximum margin criterion (LBMMC), to address the issue. We formulate the image total Laplacian matrix, image within-class Laplacian matrix and image between-class Laplacian matrix using the sample similar weight that is widely used in machine learning. The proposed LBMMC based feature extraction computes the discriminant vectors by maximizing the difference between image between-class Laplacian matrix and image within-class Laplacian matrix in both row and column directions. Experiments on the FERET and Yale face databases show the effectiveness of the proposed LBMMC based feature extraction method.     

利用标准化LDA进行人脸识别

线性判别分析（LDA）是一种较为普遍的用于特征提取的线性分类方法。提出一种基于LDA的人脸识别方法－－标准化LDA，该方法克服了传统LDA方法的缺点，重新定义了样本类间离散度矩阵，在原始定义的基础上增加一个由类间距离决定的可变权函数，使得在选择投地，能够更好地分开各个类的样本；同时，它采用一种合理而有效的方法解决矩阵奇异的问题，即保留样本类内离散度矩阵的零空间，因为这个空间包含了最具有判别能力的信息。在这个零空间里，寻找对应于样本类间离散度矩阵的较大特征值的特征向量作为最后降维的转换矩阵。实验结果显示，在人脸识别中，与传统LDA相比，该方法有更好的识别率。标准化LDA也可以用于其他图像识别问题。     

最大散度差鉴别分析及人脸识别

传统的Fisher线性鉴别分析(LDA)在人脸等高维图像识别应用中不可避免地遇到小样本问题。提出一种基于散度差准则的鉴别分析方法。与LDA方法不同的是,该方法利用样本模式的类间散布与类内散布之差而不是它们的比作为鉴别准则,这样,从根本上避免了类内散布矩阵奇异带来的困难。在ORL人脸数据库和AR人脸数据库上的实验结果验证算法的有效性。     

Speed up kernel discriminant analysis

Linear discriminant analysis (LDA) has been a popular method for dimensionality reduction, which preserves class separability. The projection vectors are commonly obtained by maximizing the between-class covariance and simultaneously minimizing the within-class covariance. LDA can be performed either in the original input space or in the reproducing kernel Hilbert space (RKHS) into which data points are mapped, which leads to kernel discriminant analysis (KDA). When the data are highly nonlinear distributed, KDA can achieve better performance than LDA. However, computing the projective functions in KDA involves eigen-decomposition of kernel matrix, which is very expensive when a large number of training samples exist. In this paper, we present a new algorithm for kernel discriminant analysis, called Spectral Regression Kernel Discriminant Analysis (SRKDA). By using spectral graph analysis, SRKDA casts discriminant analysis into a regression framework, which facilitates both efficient computation and the use of regularization techniques. Specifically, SRKDA only needs to solve a set of regularized regression problems, and there is no eigenvector computation involved, which is a huge save of computational cost. The new formulation makes it very easy to develop incremental version of the algorithm, which can fully utilize the computational results of the existing training samples. Moreover, it is easy to produce sparse projections (Sparse KDA) with a L ₁-norm regularizer. Extensive experiments on spoken letter, handwritten digit image and face image data demonstrate the effectiveness and efficiency of the proposed algorithm.     

SVM-based feature extraction for face recognition

The primary goal of linear discriminant analysis (LDA) in face feature extraction is to find an effective subspace for identity discrimination. The introduction of kernel trick has extended the LDA to nonlinear decision hypersurface. However, there remained inherent limitations for the nonlinear LDA to deal with physical applications under complex environmental factors. These limitations include the use of a common covariance function among each class, and the limited dimensionality inherent to the definition of the between-class scatter. Since these problems are inherently caused by the definition of the Fisher's criterion itself, they may not be solvable under the conventional LDA framework. This paper proposes to adopt a margin-based between-class scatter and a regularization process to resolve the issue. Essentially, we redesign the between-class scatter matrix based on the SVM margins to facilitate an effective and reliable feature extraction. This is followed by a regularization of the within-class scatter matrix. Extensive empirical experiments are performed to compare the proposed method with several other variants of the LDA method using the FERET, AR, and CMU-PIE databases.     

一种改进的线性判别分析算法在人脸识别中的应用

线性判别分析算法是一种经典的特征提取方法,但其仅在大样本情况下适用。本文针对传统线性判别分析算法面临的小样本问题和秩限制问题,提出了一种改进的线性判别分析算法ILDA。该方法在矩阵指数的基础上,重新定义了类内离散度矩阵和类间离散度矩阵,有效地同时提取类内离散度矩阵零空间和非零空间中的信息。若干人脸数据库上的比较实验表明了ILDA在人脸识别方面的有效性。     

Nonlinear discriminant mapping using the Laplacian of a graph

In this paper, an algorithm for nonlinear discriminant mapping (NDM) is presented, which elegantly integrates the ideas of both linear discriminant analysis (LDA) and Isomap by using the Laplacian of a graph. The objective of NDM is to find a linear subspace project of nonlinear data set, which preserves maximum difference between between-class scatter and within-class scatter.     

完备鉴别保局投影人脸识别算法

为了充分利用保局总体散布主元空间内的鉴别信息进行人脸识别,提出了一种完备鉴别保局投影(complete discriminant locality preserving projections,简称CDLPP)人脸识别算法.鉴于Fisher鉴别分析和保局投影已经被广泛的应用于人脸识别,完备鉴别保局投影(locality preserving projections,简称LPP)算法将这两者结合起来,分析了保局类内散布、类间散布和总体散布的主元空间和零空间内包含的鉴别信息.该算法采用奇异值分解(singular value decomposition,简称SVD),去除了不含任何鉴别信息的保局总体散布的零空间;分别在保局类内散布的主元空间和零空间提取规则鉴别特征和不规则鉴别特征;用串联的方式在特征层融合规则鉴别特征和不规则鉴别特征形成完备的鉴别特征进行人脸识别.在ORL库、FERET子库和PIE子库上的大量识别实验充分表明了完备鉴别保局投影算法的性能优于线性鉴别分析、保局投影和鉴别保局投影等现有的子空间人脸识别算法,验证了算法的有 效性.     

类依赖的线性判别分析

线性判别分析(LDA)是一种常用的特征提取方法,其目标是提取特征后样本的类间离散度和类内离散度的比值最大,即各类样本在特征空间中有最佳的可分离性.该方法利用同一个准则将所有类的样本投影到同一个特征空间中,忽略了各类样本分布特征的差异.本文提出类依赖的线性判别方法(Class-Specific LDA,CSLDA),对每一类样本寻找最优的投影矩阵,使得投影后能够更好地把该类样本与所有其他类的样本尽可能分开,并将该方法与经验核相结合,得到经验核空间中类依赖的线性判别分析.在人工数据集和UCI数据集上的实验结果表明,在输入空间和经验核空间里均有CSLDA特征提取后的识别率高于LDA.     

Extensions of LDA by PCA mixture model and class-wise features

Linear discriminant analysis (LDA) is a data discrimination technique that seeks transformation to maximize the ratio of the between-class scatter and the within-class scatter. While it has been successfully applied to several applications, it has two limitations, both concerning the underfitting problem. First, it fails to discriminate data with complex distributions since all data in each class are assumed to be distributed in the Gaussian manner. Second, it can lose class-wise information, since it produces only one transformation over the entire range of classes. We propose three extensions of LDA to overcome the above problems. The first extension overcomes the first problem by modelling the within-class scatter using a PCA mixture model that can represent more complex distribution. The second extension overcomes the second problem by taking different transformation for each class in order to provide class-wise features. The third extension combines these two modifications by representing each class in terms of the PCA mixture model and taking different transformation for each mixture component. It is shown that all our proposed extensions of LDA outperform LDA concerning classification errors for synthetic data classification, hand-written digit recognition, and alphabet recognition.     

核诱导距离度量的鲁棒判别分析

提出了基于核诱导距离度量的鲁棒判别分析算法(robust discriminant analysis based on kernel-induced distance measure,KI-RDA)。KI-RDA不仅自然地推广了线性判别分析(linear discriminant analysis,LDA),而且推广了最近提出的强有力的基于非参数最大熵的鲁棒判别分析(robust discriminant analysis based on nonparametric maximum entropy,MaxEnt-RDA)。通过采用鲁棒径向基核,KI-RDA不仅能有效处理含噪数据,而且也适合处理非高斯分布的非线性数据,其本质的鲁棒性归咎于KI-RDA通过核诱导的非欧距离代替LDA的欧氏距离来刻画类间散度和类内散度。借助这些散度,为特征提取定义类似LDA的判别准则,导致了相应的非线性优化问题。进一步借助近似策略,将优化问题转化为直接可解的广义特征值问题,由此获得降维变换(矩阵)的闭合解。最后在多类数据集上进行实验,验证了KI-RDA的有效性。由于核的多样性,使KI-RDA事实上成为了一个一般性判别分析框架。     

A multi-manifold discriminant analysis method for image feature extraction

In this paper, we propose a Multi-Manifold Discriminant Analysis (MMDA) method for an image feature extraction and pattern recognition based on graph embedded learning and under the Fisher discriminant analysis framework. In an MMDA, the within-class graph and between-class graph are, respectively, designed to characterize the within-class compactness and the between-class separability, seeking for the discriminant matrix to simultaneously maximize the between-class scatter and minimize the within-class scatter. In addition, in an MMDA, the within-class graph can represent the sub-manifold information, while the between-class graph can represent the multi-manifold information. The proposed MMDA is extensively examined by using the FERET, AR and ORL face databases, and the PolyU finger-knuckle-print databases. The experimental results demonstrate that an MMDA is effective in feature extraction, leading to promising image recognition performance.