Learning a locality discriminating projection for classification

This paper introduces a new algorithm called locality discriminating projection (LDP) for subspace learning, which provides a new scheme for discriminant analysis by considering both the manifold structure and the prior class information. In the LDP algorithm, the overlap among the class-specific manifolds is approximated by an invader graph, and a locality discriminant criterion is proposed to find the projections that best preserve the within-class local structures while decrease the between-class overlap. The feasibility of the LDP algorithm has been successfully tested in text data and visual recognition experiments. Experiment results show it is an effective technique for data modeling and classification comparing to linear discriminant analysis, locality preserving projection, and marginal Fisher analysis.     

半监督边缘判别嵌入与局部保持的维度约简

为了对高维数据进行降维处理,提出了半监督学习的边缘判别嵌入与局部保持的维度约简算法．通过最小化样本与其所属类别的中心点之间的距离,使得样本在投影子空间中能够保持其领域的拓扑结构;再通过最大化不同类别边缘间的距离,使得类别间的分离度在投影子空间中得到增强．实验结果表明：半监督边缘判别嵌入与局部保持的维度约简算法能够获得初始特征空间的较好的投影子空间．     

无关性判别保局算法及其在人脸识别中的应用

特征提取是人脸识别过程中的一个重要步骤,是人脸识别算法有效性的关键。提出了一种基于无关性判别保局的特征提取算法,并应用于人脸识别。基于保局投影算法的人 脸识别是一种有效的人脸识别算法,但它只考虑了数据的局部性,没有考虑类别信息,也没有考虑所提特征之间的相关性,现有的改进算法虽然考虑了类别信息,但是没有考虑到 类间信息。本文算法使得所提特征之间相互无关,这样降低了数据冗余,同时考虑到类别信息,使得投影后的类间区分度加强了。实验结果验证了算法的正确性和有效性,比传统 算法有较好的识别性能。     

Discriminative low-rank preserving projection for dimensionality reduction

As an effective image clustering tool, low-rank representation (LRR) can capture the intrinsic representation of the observed samples. However, firstly, the good representation does not mean good classification performance. Secondly, no projection matrix is obtained in the training stage, and it cannot deal with the new samples. By incorporating the discriminant analysis and the local neighborhood relationship of the original samples into the low-rank representation, a novel discriminative low-rank preserving projection (DLRPP) algorithm is presented for dimensionality reduction. In DLRPP, the global structure information can be captured by LRR, and the local geometricinformation is simultaneously preserved by the manifold regularization term. The constrained term is induced by the adaptive graph, which is obtained by low-rank representation coefficients. In addition, by introducing discriminant analysis constraint term, DLRPP can learn an optimal projection matrix for data dimensionality reduction. The numerous experiments on six public image datasets prove that the proposed DLRPP can obtain better recognition accuracy compared with the state-of-the-art feature extraction methods.     

间距判别投影及其在表情识别中的应用

针对全局降维方法判别信息不足,局部降维方法对邻域关系的判定存在缺陷的问题,提出一种新的基于间距的降维方法——间距判别投影（MDP）。首先,根据类的中心均值的异类近邻关系定义描述类边缘的边界向量;在这个基础上,MDP重新定义类间离散度矩阵,同时,使用全局的方法构造类内离散度矩阵;然后,MDP借鉴判别分析思想建立衡量类间距的准则,并通过类间距最大化增强样本在投影空间中的可分性。对MDP在人脸表情数据库JAFFE和Extended Cohn-Kanade上进行表情识别实验,并且跟传统的降维方法主成分分析（PCA）、最大间距准则（MMC）和边界Fisher分析（MFA）进行对比,实验结果表明,所提算法能够有效提取更具区分性的低维特征,比其他几种方法分类精度更高。     

无参数无相关最大化判别边界算法

在人脸识别算法中,无参数局部保持投影(PFLPP)是一种有效的特征提取算法, 但忽略了异类近邻样本在分类中所起的作用,并且对于近邻的处理仅利用样本与总体均值的 距离关系来判断,因此并不能有效地确定近邻关系。基于此,提出一种无参数无相关最大化 判别边界算法,有效地利用了样本的类别信息,定义了无参数同类近邻样本的相似权值与异 类近邻样本的惩罚权值,样本邻域大小可根据类内平均余弦距离和类间余弦距离自适应确定, 为了进一步增强算法的性能,给出了具有不相关性的目标函数。UMIST 和 AR 人脸库上的实 验结果表明,该算法相对于不相关保局投影分析算法和 PFLPP 算法,具有运算量低、识别性 能高的优势。     

基于奇异值分解和判别局部保持投影的多变量时间序列分类

针对现有多变量时间序列分类算法存在的要求序列等长和忽视类别信息两个不足,提出基于奇异值分解(SVD)和判别局部保持投影的分类算法。该算法基于降维思想,先通过SVD将样本的第一右奇异向量作为特征向量,以此将不等长序列转化为规模大小相同的序列;接着采用基于最大间距准则的判别局部保持投影对特征向量投影,充分利用类别信息以确保投影后同类样本尽量接近,异类样本尽量分散;最后在低维子空间采用1最近邻(1NN)、Parzen窗、支持向量机(SVM)和朴素Bayes分类器进行分类。在Australian Sign Language(ASL)、Japanese Vowels(JV)和Wafer三个公开的多变量时间序列数据集上进行的实验结果表明：在时间开销基本不变的前提下,所提方法取得了较低的分类错误率。     

结合成对约束的近邻稀疏保留投影算法

稀疏保持投影算法是一种无监督的全局线性降维方法,无法应对训练样本不足及类内样本间差异过大的情况。针对该问题,提出一种结合成对约束机制的近邻稀疏保留投影算法。利用近邻样本求取稀疏系数以保留局部结构信息,引入成对约束监督的思想,利用样本类别指导稀疏重构过程,最后定义能最大限度保留稀疏系数中蕴含的类别信息的低维子空间。将该算法用于人脸识别,实验结果证明了算法在识别率以及运行时间上的有效性和可行性。     

Effective multiplicative updates for non-negative discriminative learning in multimodal dimensionality reduction

Fisher discriminant analysis gives the unsatisfactory results if points in the same class have within-class multimodality and fails to produce the non-negativity of projection vectors. In this paper, we focus on the newly formulated within and between-class scatters based supervised locality preserving dimensionality reduction problem and propose an effective dimensionality reduction algorithm, namely, Multiplicative Updates based non-negative Discriminative Learning (MUNDL), which optimally seeks to obtain two non-negative embedding transformations with high preservation and discrimination powers for two data sets in different classes such that nearby sample pairs in the original space compact in the learned embedding space, under which the projections of the original data in different classes can be appropriately separated from each other. We also show that MUNDL can be easily extended to nonlinear dimensionality reduction scenarios by employing the standard kernel trick. We verify the feasibility and effectiveness of MUNDL by conducting extensive data visualization and classification experiments. Numerical results on some benchmark UCI and real-world datasets show the MUNDL method tends to capture the intrinsic local and multimodal structure characteristics of the given data and outperforms some established dimensionality reduction methods, while being much more efficient.     

Locality-preserved maximum information projection.

Dimensionality reduction is usually involved in the domains of artificial intelligence and machine learning. Linear projection of features is of particular interest for dimensionality reduction since it is simple to calculate and analytically analyze. In this paper, we propose an essentially linear projection technique, called locality-preserved maximum information projection (LPMIP), to identify the underlying manifold structure of a data set. LPMIP considers both the within-locality and the between-locality in the processing of manifold learning. Equivalently, the goal of LPMIP is to preserve the local structure while maximize the out-of-locality (global) information of the samples simultaneously. Different from principal component analysis (PCA) that aims to preserve the global information and locality-preserving projections (LPPs) that is in favor of preserving the local structure of the data set, LPMIP seeks a tradeoff between the global and local structures, which is adjusted by a parameter alpha, so as to find a subspace that detects the intrinsic manifold structure for classification tasks. Computationally, by constructing the adjacency matrix, LPMIP is formulated as an eigenvalue problem. LPMIP yields orthogonal basis functions, and completely avoids the singularity problem as it exists in LPP. Further, we develop an efficient and stable LPMIP/QR algorithm for implementing LPMIP, especially, on high-dimensional data set. Theoretical analysis shows that conventional linear projection methods such as (weighted) PCA, maximum margin criterion (MMC), linear discriminant analysis (LDA), and LPP could be derived from the LPMIP framework by setting different graph models and constraints. Extensive experiments on face, digit, and facial expression recognition show the effectiveness of the proposed LPMIP method.     

Unsupervised double weight graphs based discriminant analysis for dimensionality reduction

ABSTRACT

Dimensionality reduction plays an important role in pattern recognition tasks. Locality preserving projection and neighbourhood preserving embedding are popular unsupervised feature extraction methods, which try to preserve a certain local structure in the low-dimensional subspace. However, only considering the local neighbour information will limit the methods to achieve higher recognition accuracy. In this paper, an unsupervised double weight graphs based discriminant analysis method (uDWG-DA) is proposed. First, uDWG-DA considers both similar and dissimilar relationships among samples by using double weight graphs. In order to explore the dissimilar information, a new partitioning strategy is proposed to divide the data set into different clusters, where samples of different clusters are dissimilar. Then, based on L_2,1 norm, uDWG-DA finds the optimal projection to not only preserve the similar local structure but also increase the separability among different clusters of the data set. Experiments on four hyperspectral images validate the advantage and feasibility of the proposed method compared with other dimensionality reduction methods.     

Maximal local interclass embedding with application to face recognition

Dimensionality reduction of high dimensional data is involved in many problems in information processing. A new dimensionality reduction approach called maximal local interclass embedding (MLIE) is developed in this paper. MLIE can be viewed as a linear approach of a multimanifolds-based learning framework, in which the information of neighborhood is integrated with the local interclass relationships. In MLIE, the local interclass graph and the intrinsic graph are constructed to find a set of projections that maximize the local interclass scatter and the local intraclass compactness simultaneously. This characteristic makes MLIE more powerful than marginal Fisher analysis (MFA). MLIE maintains all the advantages of MFA. Moreover, the computational complexity of MLIE is less than that of MFA. The proposed algorithm is applied to face recognition. Experiments have been performed on the Yale, AR and ORL face image databases. The experimental results show that owing to the locally discriminating property, MLIE consistently outperforms up-to-date MFA, Smooth MFA, neighborhood preserving embedding and locality preserving projection in face recognition.     

局部非参数子空间分析在人脸识别中的应用

提出了一种局部非参数子空间分析算法（Local Nonparametric Subspace Analysis,LNSA）,将其应用在人脸识别中。LNSA算法结合了非参数子空间算法（Nonparametric Subspace Analysis,NSA）与局部保留投影算法（Locality Preserving Projection,LPP）。它利用LPP算法中的相似度矩阵重构NSA的类内散度矩阵,使得在最大化类间散度矩阵的同时保留了类的局部结构。在ORL人脸库和XM2VTS人脸库上作了实验并证明LNSA方法要优于其他方法。     

基于全局不相关的多流形学习

为提升人脸识别算法的鲁棒性,减少判别信息的冗余度,提出基于全局不相关的多流形判别学习算法(UFDML)。使用特征空间到特征空间的距离,学习样本局部判别信息,提出全局不相关约束,使提取的判别特征是统计不相关的。在Yale,AR,ORL人脸库上的实验结果表明,与LPP(局部保持投影)、LDA(线性判别分析)、UDP(非监督判别投影)等人脸识别算法相比,所提算法的平均识别率高于其它算法,验证了其有效性。     

完备鉴别保局投影人脸识别算法

为了充分利用保局总体散布主元空间内的鉴别信息进行人脸识别,提出了一种完备鉴别保局投影(complete discriminant locality preserving projections,简称CDLPP)人脸识别算法.鉴于Fisher鉴别分析和保局投影已经被广泛的应用于人脸识别,完备鉴别保局投影(locality preserving projections,简称LPP)算法将这两者结合起来,分析了保局类内散布、类间散布和总体散布的主元空间和零空间内包含的鉴别信息.该算法采用奇异值分解(singular value decomposition,简称SVD),去除了不含任何鉴别信息的保局总体散布的零空间;分别在保局类内散布的主元空间和零空间提取规则鉴别特征和不规则鉴别特征;用串联的方式在特征层融合规则鉴别特征和不规则鉴别特征形成完备的鉴别特征进行人脸识别.在ORL库、FERET子库和PIE子库上的大量识别实验充分表明了完备鉴别保局投影算法的性能优于线性鉴别分析、保局投影和鉴别保局投影等现有的子空间人脸识别算法,验证了算法的有 效性.     

一种监督降维的OP-LASRC算法在行为识别中的应用*

针对大数据的人体行为识别时实时性差和识别率低的问题,提出了优化投影对线性近似稀疏表示分类(OP-LASRC)的监督降维算法。OP-LASRC将高维的行为数据优化投影到低维空间,与线性近似稀疏表示(LASCR)快速分类算法相结合应用大数据的人体行为识别。首先利用LASCR的残差计算规律设计OP-LASRC算法,实现监督降维;利用线性正交投影缩减高维数据的维度,投影时减小训练样本的本类重构残差及增大类间重构残差,从而保留训练样本的类别特征。然后,对降维后的行为数据,利用LASCR算法进行分类;用L2范数估算稀疏系数,选出前k个最大的稀疏系数对应的训练样本,缩减训练样本库后用L1范数最小化和残差最小化计算得到识别结果,从识别率、鲁棒性、执行时间三个方评价此方法,在KTH行为数据库上进行实验测试。实验表明：OP-LASRC监督降维后,LASRC在分类时不仅识别率高达96.5%,执行时间比同类算法短,而且保证了强鲁棒性,证明了OP-LASRC能完美匹配LASCR算法用于行为识别,这为大数据的行为识别提供了一种新的思路。     

基于大间距准则的不相关保局投影分析

局部保持投影(Locality preserving projections,LPP)算法只保持了目标在投影后的邻域局部信息,为了更好地刻画数据的流形结构, 引入了类内和类间局部散度矩阵,给出了一种基于有效且稳定的大间距准则(Maximum margin criterion,MMC)的不相关保局投影分析方法.该方法在最大化散度矩阵迹差时,引入尺度因子α,对类内和类间局部散度矩阵进行加权,以便找到更适合分类的子空间并且可避免小样本问题; 更重要的是,大间距准则下提取的判别特征集一般情况下是统计相关的,造成了特征信息的冗余, 因此,通过增加一个不相关约束条件,利用推导出的公式提取不相关判别特征集, 这样做, 对正确识别更为有利.在Yale人脸库、PIE人脸库和MNIST手写数字库上的测试结果表明,本文方法有效且稳定, 与LPP、LDA (Linear discriminant analysis)和LPMIP(Locality-preserved maximum information projection)方法等相比,具有更高的正确识别率.     

中心近邻嵌入学习算法的人脸识别研究

针对人脸识别问题,提出了一种中心近邻嵌入的学习算法,其与经典的局部线性嵌入和保局映射不同,它是一种有监督的线性降维方法。该方法首先通过计算各类样本中心,并引入中心近邻距离代替两样本点之间的直接距离作为权系数函数的输入;然后再保持中心近邻的几何结构不变的情况下把高维数据嵌入到低维坐标系中。通过中心近邻嵌入学习算法与其他3种人脸识别方法(即主成分分析、线形判别分析及保局映射)在ORL、Yale及UMIST人脸库上进行的比较实验结果表明,它在高维数据低维可视化和人脸识别效果等方面均较其他3种方法取得了更好的效果。     

一种用于特征提取的保局判别分析算法

针对保局投影(LPP)为无监督算法的局限,提出了一种新的监督版的LPP,即保局判别分析(LPDA)算法。LPDA吸收了流形学习算法与最大边界准则(MMC)的共同特点,可以将高维的人脸数据投影到低维子空间,具有能处理新样本与无小样本问题的优点。与现有的多种经典相关方法相比,从Yale, UMIST及MIT 3个人脸数据库的实验结果表明,提出的LPDA算法在降维的同时提取了用于人脸识别的更有效的特征,人脸图像识别性能较好,具有较强的判别分析能力。     

基于L1范数的分块二维局部保持投影算法

针对高维输入数据维数较大时可能存在奇异值问题,同时为提高算法的运算效率以及算法的鲁棒性,提出了一种基于L1范数的分块二维局部保持投影算法B2DLPP-L1。传统的局部保持投影算法为避免出现奇异值问题,首先运用主成分分析算法将高维数据投影到子空间中,然而这种方式将会造成高维数据中部分有效信息的流失,B2DLPP-L1算法选择将二维数据直接作为输入数据,避免运用向量形式的输入数据时可能造成的数据流失;同时该算法对二维输入数据进行分块处理,将分块后的数据块作为新的输入数据,之后运用基于L1范数的二维局部保持投影算法对其进行降维。理论上,B2DLPP-L1算法能够较好地对数据进行降维,不仅能够保持高维数据中的有效信息,降低计算复杂程度,提高算法的运行效率,同时还能够克服存在外点情况下分类准确率较低问题,提高算法的鲁棒性。通过选择不同的人脸数据库进行实验,实验结果表明,在存在外点的情况下,运用最近邻分类器时能够取得更高的分类准确率,同时所需的分类时间有所减少。