流形距离与压缩感知核稀疏投影的局部线性嵌入算法

流形学习算法的目的是发现嵌入在高维数据空间中的低维表示,现有的流形学习算法对邻域参数k和噪声比较敏感。针对此问题,文中提出一种流形距离与压缩感知核稀疏投影的局部线性嵌入算法,其核心思想是集成局部线性嵌入算法对高维流形结构数据的降维有效性与压缩感知核稀疏投影的强鉴别性,以实现高效有降噪流形学习。首先,在选择各样本点的近邻域时,采用流形距离代替欧氏距离度量数据间相似度的方法,创建能够正确反映流形内部结构的邻域图,解决以欧氏距离作为相似性度量时对邻域参数的敏感。其次,利用压缩感知核稀疏投影作为从高维观测空间到低维嵌入空间的映射,增强算法的鉴别性。最后,利用Matlab工具对实验数据集进行仿真,进一步验证所提算法的有效性。     

最小距离鉴别投影及其在人脸识别中的应用

针对人脸识别问题,提出了最小距离鉴别投影算法,其与经典的线性鉴别分析不同,它是一种流形学习降维算法。该算法首先定义样本的类内相似度与类间相似度:前者能够度量样本与类内中心的距离关系,后者不仅能够反映样本与类间中心的距离关系而且能够反映样本类间距与类内距的大小关系;然后将高维数据映射到低维特征空间,使得样本到类内中心距离最小同时到类间中心距离最大。最后,在ORL、FERET及AR人脸库上的实验结果表明所提算法识别性能要优于其他算法。     

改进的局部线性嵌入算法及其应用

局部线性嵌入算法（LLE）中常用欧氏距离来度量样本间相似度,而对于具有低维流形结构的高维数据,欧氏距离不能衡量流形上两点间相对位置关系。提出基于Geodesic Rank-order距离的局部线性嵌入算法（简称GRDLLE）。应用最短路径算法（Dijkstra算法）找到最短路径长度来近似计算任意两个样本间的测地线距离,计算Rank-order距离用于LLE算法的相似性度量。将GRDLLE算法、其他改进LLE的流形学习算法及2DPCA算法在ORL与Yale数据集上进行对比实验,对数据用GRDLLE算法进行降维后人脸识别率有所提高,结果表明GRDLLE算法具有很好的降维效果。     

基于测地线距离的广义高斯型Laplacian 特征映射

传统的Laplacian 特征映射是基于欧氏距离的近邻数据点的保持,近邻的高维数据点映射到内在低维空间后仍为近邻点,高维数据点的近邻选取最终将影响全局低维坐标.将测地线距离和广义高斯函数融合到传统的Laplacian 特征映射算法中,首先提出了一种基于测地线距离的广义高斯型Laplacian 特征映射算法(geodesicdistance-based generalized Gaussian LE,简称GGLE),该算法在用不同的广义高斯函数度量高维数据点间的相似度时,获得的全局低维坐标呈现出不同的聚类特性;然后,利用这种特性进一步提出了它的集成判别算法,该集成判别算法的主要优点是:近邻参数K 固定,邻接图和测地线距离矩阵都只构造一次.在木纹数据集上的识别实验结果表明,这是一种有效的基于流形的集成判别算法.     

基于局部线性逼近的流形学习算法

流形学习方法是根据流形的定义提出的一种非线性数据降维方法,主要思想是发现嵌入在高维数据空间的低维光滑流形.局部线性嵌入算法是应用比较广泛的一种流形学习方法,传统的局部线性嵌入算法的一个主要缺点就是在处理稀疏源数据时会失效,而实际应用中很多情况还要面对处理源数据稀疏的问题.在分析局部线性嵌入算法的基础上提出了基于局部线性逼近思想的流形学习算法,其通过采用直接估计梯度值的方法达到局部线性逼近的目的,从而实现高维非线性数据的维数约简,最后在S-曲线上进行稀疏采样测试取得良好降维效果.     

LLE方法的分类与研究

对于低维数据的分类很常见, 但是对于高维数据的分类却不多, 主要是因为维度太高. 尤其对于分布不均匀的样本集, 传统的局部线性嵌入算法易受到近邻点个数的影响, 为了克服这一问题, 提出改进距离的局部线性嵌入算法. 通过实验表明, 改进距离的局部线性嵌入算法能使原来的样本集尽可能的分布均匀, 从而降低近邻点个数的取值对局部线性嵌入的影响, 在保证分类准确的前提下, 达到了有效缩短时间的目的.     

在高维数据上的近邻传播聚类降维研究北大核心CSCD

为了使近邻传播(AP)聚类在高维空间中获得更好的聚类效果,该文提出一种基于谱分析的近邻传播聚类方法(Affinity Propagation based on Spectrum analyze,AP-SA)。首先,通过采用谱分析技术将分布在高维非线性的数据点集映射到几乎线性的子空间上,映射过程实现高维数据降至低维。最后,通过AP聚类算法对映射在低维空间上的数据进行聚类,从而提高了AP算法在高维空间上的聚类性能。仿真实验结果表明,该方法相比于传统AP算法,在低维数据中无明显的优势,但随着实验的数据集的样本规模与维数的增加,在高维数据中的该方法降低了聚类时间的同时,也保证了较好的聚类效果。     

基于随机游走扩散映射的降维算法

传统数据降维算法分为线性或流形学习降维算法,但在实际应用中很难确定需要哪一类算法.设计一种综合的数据降维算法,以保证它的线性降维效果下限为主成分分析方法且在流形学习降维方面能揭示流形的数据结构.通过对高维数据构造马尔可夫转移矩阵,使越相似的节点转移概率越大,从而发现高维数据降维到低维流形的映射关系.实验结果表明,在人造...     

一种在源数据稀疏情况下的数据降维算法

流形学习方法是根据流形的定义提出的一种非线性数据降维方法,主要思想是发现嵌入在高维数据空间的低维光滑流形。从分析基于流形学习理论的局部线性嵌入算法入手,针对传统的局部线性嵌入算法在源数据稀疏时会失效的缺点,提出了基于局部线性逼近思想的流形学习算法,并在S-曲线上采样测试取得良好降维效果。     

基于流形学习的维数约简算法

介绍线性维数约简的主成分分析和多维尺度算法,描述几种经典的能发现嵌入在高维数据空间的低维光滑流形非线性维数约简算法,包括等距映射、局部线性嵌入、拉普拉斯特征映射、局部切空间排列、最大方差展开。与线性维数约简算法相比,非线性维数约简算法通过维数约简能够发现不同类型非线性高维数据的本质特征。     

Locally Minimizing Embedding and Globally Maximizing Variance: Unsupervised Linear Difference Projection for Dimensionality Reduction

Recently, many dimensionality reduction algorithms, including local methods and global methods, have been presented. The representative local linear methods are locally linear embedding (LLE) and linear preserving projections (LPP), which seek to find an embedding space that preserves local information to explore the intrinsic characteristics of high dimensional data. However, both of them still fail to nicely deal with the sparsely sampled or noise contaminated datasets, where the local neighborhood structure is critically distorted. On the contrary, principal component analysis (PCA), the most frequently used global method, preserves the total variance by maximizing the trace of feature variance matrix. But PCA cannot preserve local information due to pursuing maximal variance. In order to integrate the locality and globality together and avoid the drawback in LLE and PCA, in this paper, inspired by the dimensionality reduction methods of LLE and PCA, we propose a new dimensionality reduction method for face recognition, namely, unsupervised linear difference projection (ULDP). This approach can be regarded as the integration of a local approach (LLE) and a global approach (PCA), so that it has better performance and robustness in applications. Experimental results on the ORL, YALE and AR face databases show the effectiveness of the proposed method on face recognition.     

Supervised optimal locality preserving projection

In the past few years, the computer vision and pattern recognition community has witnessed a rapid growth of a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among these methods, locality preserving projection (LPP) is one of the most promising feature extraction techniques. Unlike the unsupervised learning scheme of LPP, this paper follows the supervised learning scheme, i.e. it uses both local information and class information to model the similarity of the data. Based on novel similarity, we propose two feature extraction algorithms, supervised optimal locality preserving projection (SOLPP) and normalized Laplacian-based supervised optimal locality preserving projection (NL-SOLPP). Optimal here means that the extracted features via SOLPP (or NL-SOLPP) are statistically uncorrelated and orthogonal. We compare the proposed SOLPP and NL-SOLPP with LPP, orthogonal locality preserving projection (OLPP) and uncorrelated locality preserving projection (ULPP) on publicly available data sets. Experimental results show that the proposed SOLPP and NL-SOLPP achieve much higher recognition accuracy.     

Sparsity preserving projections with applications to face recognition

Dimensionality reduction methods (DRs) have commonly been used as a principled way to understand the high-dimensional data such as face images. In this paper, we propose a new unsupervised DR method called sparsity preserving projections (SPP). Unlike many existing techniques such as local preserving projection (LPP) and neighborhood preserving embedding (NPE), where local neighborhood information is preserved during the DR procedure, SPP aims to preserve the sparse reconstructive relationship of the data, which is achieved by minimizing a L1 regularization-related objective function. The obtained projections are invariant to rotations, rescalings and translations of the data, and more importantly, they contain natural discriminating information even if no class labels are provided. Moreover, SPP chooses its neighborhood automatically and hence can be more conveniently used in practice compared to LPP and NPE. The feasibility and effectiveness of the proposed method is verified on three popular face databases (Yale, AR and Extended Yale B) with promising results.     

基于判别正则化的局部保留投影

局部保留投影（Locality preserving projections,LPP）是一种常用的线性化流形学习方法,其通过线性嵌入来保留基于图所描述的流形数据本质结构特征,因此LPP对图的依赖性强,且在嵌入过程中缺少对图描述的进一步分析和挖掘。当图对数据本质结构特征描述不恰当时,LPP在嵌入过程中不易实现流形数据本质结构的有效提取。为了解决这个问题,本文在给定流形数据图描述的条件下,通过引入局部相似度阈值进行局部判别分析,并据此建立判别正则化局部保留投影（简称DRLPP）。该方法能够在现有图描述的条件下,有效突出不同流形结构在线性嵌入空间中的可分性。在人造合成数据集和实际标准数据集上对DRLPP以及相关算法进行对比实验,实验结果证明了DRLPP的有效性。     

Kernel class-wise locality preserving projection

In the recent years, the pattern recognition community paid more attention to a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among them, locality preserving projection (LPP) is one of the most promising feature extraction techniques. However, when LPP is applied to the classification tasks, it shows some limitations, such as the ignorance of the label information. In this paper, we propose a novel local structure based feature extraction method, called class-wise locality preserving projection (CLPP). CLPP utilizes class information to guide the procedure of feature extraction. In CLPP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and the class information. The kernelized (nonlinear) counterpart of this linear feature extractor is also established in the paper. Moreover, a kernel version of CLPP namely Kernel CLPP (KCLPP) is developed through applying the kernel trick to CLPP to increase its performance on nonlinear feature extraction. Experiments on ORL face database and YALE face database are performed to test and evaluate the proposed algorithm.     

Face recognition using laplacianfaces

We propose an appearance-based face recognition method called the Laplacianface approach. By using locality preserving projections (LPP), the face images are mapped into a face subspace for analysis. Different from principal component analysis (PCA) and linear discriminant analysis (LDA) which effectively see only the Euclidean structure of face space, LPP finds an embedding that preserves local information, and obtains a face subspace that best detects the essential face manifold structure. The Laplacianfaces are the optimal linear approximations to the eigenfunctions of the Laplace Beltrami operator on the face manifold. In this way, the unwanted variations resulting from changes in lighting, facial expression, and pose may be eliminated or reduced. Theoretical analysis shows that PCA, LDA, and LPP can be obtained from different graph models. We compare the proposed Laplacianface approach with Eigenface and Fisherface methods on three different face data sets. Experimental results suggest that the proposed Laplacianface approach provides a better representation and achieves lower error rates in face recognition.     

Local structure based supervised feature extraction

In the past few years, the computer vision and pattern recognition community has witnessed the rapid growth of a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among them, locality preserving projection (LPP) is one of the most promising feature extraction techniques. However, when LPP is applied to the classification tasks, it shows some limitations, such as the ignorance of the label information. In this paper, we propose a novel feature extraction method, called locally discriminating projection (LDP). LDP utilizes class information to guide the procedure of feature extraction. In LDP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and the class information. The similarity has several good properties which help to discover the true intrinsic structure of the data, and make LDP a robust technique for the classification tasks. We compare the proposed LDP approach with LPP, as well as other feature extraction methods, such as PCA and LDA, on the public available data sets, FERET and AR. Experimental results suggest that LDP provides a better representation of the class information and achieves much higher recognition accuracies.     

Local sparse representation projections for face recognition

How to define the sparse affinity weight matrices is still an open problem in existing manifold learning algorithm. In this paper, we propose a novel supervised learning method called local sparse representation projections (LSRP) for linear dimensionality reduction. Differing from sparsity preserving projections (SPP) and the recent manifold learning methods such as locality preserving projections (LPP), LSRP introduces the local sparse representation information into the objective function. Although there are no labels used in the local sparse representation, it still can provide better measure coefficients and significant discriminant abilities. By combining the local interclass neighborhood relationships and sparse representation information, LSRP aims to preserve the local sparse reconstructive relationships of the data and simultaneously maximize the interclass separability. Comprehensive comparison and extensive experiments show that LSRP achieves higher recognition rates than principle component analysis, linear discriminant analysis and the state-of-the-art techniques such as LPP, SPP and maximum variance projections.     

基于全局不相关的多流形学习

为提升人脸识别算法的鲁棒性,减少判别信息的冗余度,提出基于全局不相关的多流形判别学习算法(UFDML)。使用特征空间到特征空间的距离,学习样本局部判别信息,提出全局不相关约束,使提取的判别特征是统计不相关的。在Yale,AR,ORL人脸库上的实验结果表明,与LPP(局部保持投影)、LDA(线性判别分析)、UDP(非监督判别投影)等人脸识别算法相比,所提算法的平均识别率高于其它算法,验证了其有效性。     

Complete neighborhood preserving embedding for face recognition

Neighborhood preserving embedding (NPE) is a linear approximation to the locally linear embedding algorithm which can preserve the local neighborhood structure on the data manifold. However, in typical face recognition where the number of data samples is smaller than the dimension of data space, it is difficult to directly apply NPE to high dimensional matrices because of computational complexity. Moreover, in such case, NPE often suffers from the singularity problem of eigenmatrix, which makes the direct implementation of the NPE algorithm almost impossible. In practice, principal component analysis or singular value decomposition is applied as a preprocessing step to attack these problems. Nevertheless, this strategy may discard dimensions that contain important discriminative information and the eigensystem computation of NPE could be unstable. Towards a practical dimensionality reduction method for face data, we develop a new scheme in this paper, namely, the complete neighborhood preserving embedding (CNPE). CNPE transforms the singular generalized eigensystem computation of NPE into two eigenvalue decomposition problems. Moreover, a feasible and effective procedure is proposed to alleviate the computational burden of high dimensional matrix for typical face image data. Experimental results on the ORL face database and the Yale face database show that the proposed CNPE algorithm achieves better performance than other feature extraction methods, such as Eigenfaces, Fisherfaces and NPE, etc.