Feature extraction based on fuzzy class mean embedding (FCME) with its application to face and palm biometrics

In the local discriminant embedding (LDE) framework, the neighbor and class of data points were used to construct the graph embedding for classification problems. From a high-dimensional to a low-dimensional subspace, data points of the same class maintain their intrinsic neighbor relations, whereas neighboring data points of different classes no longer stick to one another. However, face images are always affected by variations in illumination conditions and different facial expressions in the real world. So, distant data points are not deemphasized efficiently by LDE and it may degrade the performance of classification. In order to solve above problems, in this paper, we investigate the fuzzy set theory and class mean of LDE, called fuzzy class mean embedding (FCME), using the fuzzy k-nearest neighbor (FKNN) and the class sample average to enhance its discriminant power in their mapping into a low dimensional space. In the proposed method, a membership degree matrix is firstly calculated using FKNN, then the membership degree and class mean are incorporated into the definition of the Laplacian scatter matrix. The optimal projections of FCME can be obtained by solving a generalized eigenfunction. Experimental results on the Wine dataset, ORL, Yale, AR, FERET face database and PolyU palmprint database show the effectiveness of the proposed method.     

正交化近邻关系保持的降维及分类算法

针对近邻关系保持嵌入（NPE）算法易于受到降低后的维数影响,而且性能依赖于正确的维数估计的问题,提出了一种正交化的近邻关系保持的嵌入降维方法——ONPE。ONPE方法是使用数据点间的近邻关系来构造邻接图,假设每个数据点都能由其近邻点的线性组合表示,则可以通过提取数据点的局部几何信息,并在降维中保持提取的局部几何信息,迭代地计算正交基来得到数据的低维嵌入坐标。同时,在ONPE算法的基础上,利用局部几何信息,提出了一种在低维空间中使用标签传递（LNP）的分类算法——ONPC。其是假设高维空间中的局部近邻关系在降维后的空间中依然得到保持,并且数据点的类别可由近邻点的类别得到。在人工数据和人脸数据上的实验表明,该算法在减少维数依赖的同时,能有效提高NPE算法的分类性能。     

Discriminant neighborhood embedding for classification

In this paper a novel subspace learning method called discriminant neighborhood embedding (DNE) is proposed for pattern classification. We suppose that multi-class data points in high-dimensional space tend to move due to local intra-class attraction or inter-class repulsion and the optimal embedding from the point of view of classification is discovered consequently. After being embedded into a low-dimensional subspace, data points in the same class form compact submanifod whereas the gaps between submanifolds corresponding to different classes become wider than before. Experiments on the UMIST and MNIST databases demonstrate the effectiveness of our method.     

间距判别投影及其在表情识别中的应用

针对全局降维方法判别信息不足,局部降维方法对邻域关系的判定存在缺陷的问题,提出一种新的基于间距的降维方法——间距判别投影（MDP）。首先,根据类的中心均值的异类近邻关系定义描述类边缘的边界向量;在这个基础上,MDP重新定义类间离散度矩阵,同时,使用全局的方法构造类内离散度矩阵;然后,MDP借鉴判别分析思想建立衡量类间距的准则,并通过类间距最大化增强样本在投影空间中的可分性。对MDP在人脸表情数据库JAFFE和Extended Cohn-Kanade上进行表情识别实验,并且跟传统的降维方法主成分分析（PCA）、最大间距准则（MMC）和边界Fisher分析（MFA）进行对比,实验结果表明,所提算法能够有效提取更具区分性的低维特征,比其他几种方法分类精度更高。     

Local uncorrelated local discriminant embedding for face recognition

The feature extraction algorithm plays an important role in face recognition. However, the extracted features also have overlapping discriminant information. A property of the statistical uncorrelated criterion is that it eliminates the redundancy among the extracted discriminant features, while many algorithms generally ignore this property. In this paper, we introduce a novel feature extraction method called local uncorrelated local discriminant embedding (LULDE). The proposed approach can be seen as an extension of a local discriminant embedding (LDE) framework in three ways. First, a new local statistical uncorrelated criterion is proposed, which effectively captures the local information of interclass and intraclass. Second, we reconstruct the affinity matrices of an intrinsic graph and a penalty graph, which are mentioned in LDE to enhance the discriminant property. Finally, it overcomes the small-sample-size problem without using principal component analysis to preprocess the original data, which avoids losing some discriminant information. Experimental results on Yale, ORL, Extended Yale B, and FERET databases demonstrate that LULDE outperforms LDE and other representative uncorrelated feature extraction methods.     

基于随机子空间的局部鉴别投影*

针对高维数据容易对噪声敏感及容易造成维数灾难问题,文中提出基于随机子空间的局部鉴别投影算法(RSLDP).利用随机子空间方法对高维的原始数据进行特征选择,在生成的低维特征子空间构造近邻图,降低噪声影响.RSLDP通过最大化局部类间加权散度和最小化局部类内加权散度,同时最小化样本的总体局部散度,改进局部最大间距鉴别嵌入算法,较好刻画样本与其类间类内近邻中心点的关系,有利于鉴别特征的提取.在CMU PIE和AR这2个人脸数据库上的实验表明文中算法的有效性.     

基于最近邻的随机非线性降维

针对线性降维技术应用于具有非线性结构的数据时无法得到令人满意的结果的问题,提出一种新的着重于保持高维空间局部最近邻信息的非线性随机降维算法(NNSE)。该算法首先在高维空间中通过计算样本点之间的欧氏距离找出每个样本点的最近邻点,接着在低维空间中产生一个随机的初始分布;然后通过将低维空间中的样本点不断向其最近邻点的平均位置移动,直到产生稳定的低维嵌入结果。与一种先进的非线性随机降维算法——t分布随机邻域嵌入(t-SNE)相比,NNSE算法得到的低维结果在可视化方面与t-SNE算法相差不大,但通过比较两者的量化指标可以发现,NNSE算法在保持最近邻信息方面上明显优于t-SNE算法。     

基于二维邻域保持判别嵌入的人脸识别

提出二维邻域保持判别嵌入(2DNPDE)算法,该算法是一种有监督的基于二维图像矩阵的特征提取算法.为表示样本的类内邻域结构和类间距离关系,分别构建类内邻接矩阵和类间相似度矩阵.2DNPDE所获得的投影空间不但使不同类数据点的低维嵌入相互分离,而且保留同类样本的邻域结构和不同类样本的距离关系.在ORL和AR人脸数据库上的实验表明,该算法具有更好的识别效果.     

面向文档分类的LDE和简化SVM方法研究

为了快速准确地对文档进行分类,提出了一种基于局部鉴别嵌入LDE和简化SVM的高效文档分类算法。该算法首先利用LDE算法把高维文档数据投影到低维特征空间,然后在低维特征空间利用精简SVM进行分类。实验结果表明该算法具有分类准确率高和运行速度快的优点。     

保持全局和局部特性的黎曼流形改进算法

黎曼流形学习(RML)是一种全局算法,但其不能较好地保持数据局部邻域的几何性质。为解决这个问题,提出一种基于黎曼流形学习（RML）的多结构算法。先对数据集进行主成分分析（PCA）投影,再构造邻域图,然后把整个数据集分为两个部分求低维嵌入坐标,对于基准点的k近邻,采用能保持其和近邻点局部性质的权值矩阵得到低维嵌入;对于其他点仍采用RML算法,使其达到既能维持数据点的全局结构,又能最大限度地保持其局部几何性质的目的。实验结果验证了该算法的有效性和实时性。     

Supervised Discriminant Projection with Its Application to Face Recognition

In the past few decades, many face recognition methods have been developed. Among these methods, subspace analysis is an effective approach for face recognition. Unsupervised discriminant projection (UDP) finds an embedding subspace that preserves local structure information, and uncovers and separates embedding corresponding to different manifolds. Though UDP has been applied in many fields, it has limits to solve the classification tasks, such as the ignorance of the class information. Thus, a novel subspace method, called supervised discriminant projection (SDP), is proposed for face recognition in this paper. In our method, the class information was utilized in the procedure of feature extraction. In SDP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and class information. We test the performance of the proposed method SDP on three popular face image databases (i.e. AR database, Yale database, and a subset of FERET database). Experimental results show that the proposed method is effective.     

一种新的基于MMC和LSE的监督流形学习算法

针对局部样条嵌入算法 （Local spline embedding,LSE） 存在样本外点学习和无监督模式学习问题,本文提出了一种新颖的正交局部样条判别投影算法 （O-LSDP）.该算法通过引入明确的线性映射关系,构建平移缩放模型,以及正交化特征子空间,从而使该算法能够应用于模式分类问题并显著改善了算法的分类识别能力.在标准人 脸数据库和植物叶片数据库上的实验结果验证了该算法的有效性与可行性.     

Effective multiplicative updates for non-negative discriminative learning in multimodal dimensionality reduction

Fisher discriminant analysis gives the unsatisfactory results if points in the same class have within-class multimodality and fails to produce the non-negativity of projection vectors. In this paper, we focus on the newly formulated within and between-class scatters based supervised locality preserving dimensionality reduction problem and propose an effective dimensionality reduction algorithm, namely, Multiplicative Updates based non-negative Discriminative Learning (MUNDL), which optimally seeks to obtain two non-negative embedding transformations with high preservation and discrimination powers for two data sets in different classes such that nearby sample pairs in the original space compact in the learned embedding space, under which the projections of the original data in different classes can be appropriately separated from each other. We also show that MUNDL can be easily extended to nonlinear dimensionality reduction scenarios by employing the standard kernel trick. We verify the feasibility and effectiveness of MUNDL by conducting extensive data visualization and classification experiments. Numerical results on some benchmark UCI and real-world datasets show the MUNDL method tends to capture the intrinsic local and multimodal structure characteristics of the given data and outperforms some established dimensionality reduction methods, while being much more efficient.     

Local Tangent Space Discriminant Analysis

We propose a novel supervised dimensionality reduction method named local tangent space discriminant analysis (TSD) which is capable of utilizing the geometrical information from tangent spaces. The proposed method aims to seek an embedding space where the local manifold structure of the data belonging to the same class is preserved as much as possible, and the marginal data points with different class labels are better separated. Moreover, TSD has an analytic form of the solution and can be naturally extended to non-linear dimensionality reduction through the kernel trick. Experimental results on multiple real-world data sets demonstrate the effectiveness of the proposed method.     

基于KL散度和近邻点间距离的球面嵌入算法

针对现有球面嵌入算法在非近邻点间的距离度量不准确或缺失的情况下,不能有效地进行低维嵌入的问题,提出了一种新的球面嵌入算法,它能够只利用近邻点间的距离,将任何尺度的高维数据嵌入到单位球面上,同时求出适合原始数据分布的球面半径。该算法从一个随机产生的球面分布开始,利用KL散度衡量每对近邻点间的归一化距离在原始空间和球面空间中的差异,并基于此差异构建出目标函数,然后再用带有动量的随机梯度下降法,不断优化球面上点的分布,直到结果稳定。为了测试算法,模拟产生了两类球面分布数据：分别是球面均匀分布和球面正态分布的数据。实验结果表明,对于球面均匀分布的数据,即使在近邻点个数很少的情况下,仍然能够将数据准确地嵌入球面空间,嵌入后的数据分布与原始数据分布的均方根误差（RMSE）低于0.00001,且球面半径的估算误差低于0.000001;而对于球面正态分布的数据,在近邻点个数较多的情况下,该算法也可以将数据较准确地嵌入球面空间。因此,在非近邻点间距离缺失的情况下,所提方法仍然可以较准确地对数据进行低维嵌入,这非常有利于数据的可视化研究。     

完备自适应近邻图嵌入的局部鉴别投影算法

针对基于自适应近邻图嵌入的局部鉴别投影算法（Neighborhood graph embedding based local adaptive discriminant analysis, LADP ）仅仅利用局部类内离差矩阵主元空间的鉴别信息而丢失了其零空间内大量鉴别信息的不足,结合全空间的基本思想提出了完备的基于自适应近邻图嵌入的局部鉴别投影算法（ Complete LADP,CLADP）。在局部类内离差矩阵的零空间内,通过最大化局部类间离差矩阵提取不规则鉴别特征,在局部类间离差矩阵的主元空间内,通过最大化局部类间离差矩阵的同时最小化局部类 内离差矩阵提取规则鉴别特征,最后将不规则鉴别特征和规则鉴别特征串联形成CLADP特征。在ORL,Yale以及PIE人脸库上的人脸识别实验结果证明了CLADP的有效性。     

Locality preserving embedding for face and handwriting digital recognition

Most supervised manifold learning-based methods preserve the original neighbor relationships to pursue the discriminating power. Thus, structure information of the data distributions might be neglected and destroyed in low-dimensional space in a certain sense. In this paper, a novel supervised method, called locality preserving embedding (LPE), is proposed to feature extraction and dimensionality reduction. LPE can give a low-dimensional embedding for discriminative multi-class sub-manifolds and preserves principal structure information of the local sub-manifolds. In LPE framework, supervised and unsupervised ideas are combined together to learn the optimal discriminant projections. On the one hand, the class information is taken into account to characterize the compactness of local sub-manifolds and the separability of different sub-manifolds. On the other hand, at the same time, all the samples in the local neighborhood are used to characterize the original data distributions and preserve the structure in low-dimensional subspace. The most significant difference from existing methods is that LPE takes the distribution directions of local neighbor data into account and preserves them in low-dimensional subspace instead of only preserving the each local sub-manifold’s original neighbor relationships. Therefore, LPE optimally preserves both the local sub-manifold’s original neighborhood relationships and the distribution direction of local neighbor data to separate different sub-manifolds as far as possible. The criterion, similar to the classical Fisher criterion, is a Rayleigh quotient in form, and the optimal linear projections are obtained by solving a generalized Eigen equation. Furthermore, the framework can be directly used in semi-supervised learning, and the semi-supervised LPE and semi-supervised kernel LPE are given. The proposed LPE is applied to face recognition (on the ORL and Yale face databases) and handwriting digital recognition (on the USPS database). The experimental results show that LPE consistently outperforms classical linear methods, e.g., principal component analysis and linear discriminant analysis, and the recent manifold learning-based methods, e.g., marginal Fisher analysis and constrained maximum variance mapping.     

An optimization criterion for generalized marginal Fisher analysis on undersampled problems

Marginal Fisher analysis (MFA) not only aims to maintain the original relations of neighboring data points of the same class but also wants to keep away neighboring data points of the different classes. MFA can effectively overcome the limitation of linear discriminant analysis (LDA) due to data distribution assumption and available projection directions. However, MFA confronts the undersampled problems. Generalized marginal Fisher analysis (GMFA) based on a new optimization criterion is presented, which is applicable to the undersampled problems. The solutions to the proposed criterion for GMFA are derived, which can be characterized in a closed form. Among the solutions, two specific algorithms, namely, normal MFA (NMFA) and orthogonal MFA (OMFA), are studied, and the methods to implement NMFA and OMFA are proposed. A comparative study on the undersampled problem of face recognition is conducted to evaluate NMFA and OMFA in terms of classification accuracy, which demonstrates the effectiveness of the proposed algorithms.     

A Road Network Embedding Technique for K-Nearest Neighbor Search in Moving Object Databases

A very important class of queries in GIS applications is the class of K-nearest neighbor queries. Most of the current studies on the K-nearest neighbor queries utilize spatial index structures and hence are based on the Euclidean distances between the points. In real-world road networks, however, the shortest distance between two points depends on the actual path connecting the points and cannot be computed accurately using one of the Minkowski metrics. Thus, the Euclidean distance may not properly approximate the real distance. In this paper, we apply an embedding technique to transform a road network to a high dimensional space in order to utilize computationally simple Minkowski metrics for distance measurement. Subsequently, we extend our approach to dynamically transform new points into the embedding space. Finally, we propose an efficient technique that can find the actual shortest path between two points in the original road network using only the embedding space. Our empirical experiments indicate that the Chessboard distance metric (L) in the embedding space preserves the ordering of the distances between a point and its neighbors more precisely as compared to the Euclidean distance in the original road network.     

小波核局部Fisher判别分析的高光谱遥感影像特征提取

为了提高高光谱遥感影像的分类精度,充分利用影像的光谱和局部信息,文中提出小波核局部Fisher判别分析的高光谱遥感影像特征提取方法.通过小波核函数将数据集从低维原始空间映射至高维特征空间,考虑到数据的局部信息,利用加权矩阵计算散度矩阵,对局部Fisher判别准则函数求解最优特征矩阵,使不同类别的样本在高维特征空间中的可分离性更佳.在2个公开高光谱数据集上的实验表明,文中方法的总体分类精度和Kappa系数都有所提高.