Subspace manifold learning with sample weights

Subspace manifold learning represents a popular class of techniques in statistical image analysis and object recognition. Recent research in the field has focused on nonlinear representations; locally linear embedding (LLE) is one such technique that has recently gained popularity. We present and apply a generalization of LLE that introduces sample weights. We demonstrate the application of the technique to face recognition, where a model exists to describe each face’s probability of occurrence. These probabilities are used as weights in the learning of the low-dimensional face manifold. Results of face recognition using this approach are compared against standard nonweighted LLE and PCA. A significant improvement in recognition rates is realized using weighted LLE on a data set where face occurrences follow the modeled distribution.     

Discriminant locally linear embedding with high-order tensor data.

Graph-embedding along with its linearization and kernelization provides a general framework that unifies most traditional dimensionality reduction algorithms. From this framework, we propose a new manifold learning technique called discriminant locally linear embedding (DLLE), in which the local geometric properties within each class are preserved according to the locally linear embedding (LLE) criterion, and the separability between different classes is enforced by maximizing margins between point pairs on different classes. To deal with the out-of-sample problem in visual recognition with vector input, the linear version of DLLE, i.e., linearization of DLLE (DLLE/L), is directly proposed through the graph-embedding framework. Moreover, we propose its multilinear version, i.e., tensorization of DLLE, for the out-of-sample problem with high-order tensor input. Based on DLLE, a procedure for gait recognition is described. We conduct comprehensive experiments on both gait and face recognition, and observe that: 1) DLLE along its linearization and tensorization outperforms the related versions of linear discriminant analysis, and DLLE/L demonstrates greater effectiveness than the linearization of LLE; 2) algorithms based on tensor representations are generally superior to linear algorithms when dealing with intrinsically high-order data; and 3) for human gait recognition, DLLE/L generally obtains higher accuracy than state-of-the-art gait recognition algorithms on the standard University of South Florida gait database.     

Modified locally linear discriminant embedding for plant leaf recognition

Based on locally linear embedding (LLE) and modified maximizing margin criterion (MMMC), a modified locally linear discriminant embedding (MLLDE) algorithm is proposed for plant leaf recognition in this paper. By MLLDE, the plant leaf images are mapped into a leaf subspace for analysis, which can detect the essential leaf manifold structure. Furthermore, the unwanted variations resulting from changes in period, location, and illumination can be eliminated or reduced. Different from principal component analysis (PCA) and linear discriminant analysis (LDA), which can only deal with flat Euclidean structures of plant leaf space, MLLDE not only inherits the advantages of locally linear embedding (LLE), but makes full use of class information to improve discriminant power by introducing translation and rescaling models. The experimental results on real plant leaf database show that the MLLDE is effective for plant leaf recognition.     

基于局部线性嵌入和最近特征线的人耳识别

针对人耳生物特征,通过分析早期人耳识别方法的不足,提出了一种局部线性嵌入（LLE）和最近特征线（NFL）相结合的人耳识别方法。首先依据流形学习思想,采用局部线性嵌入算法提取人耳图像特征,然后采用最近特征线分类器进行人耳识别。实验结果表明,该方法在人耳姿态变化时能够取得非常理想的识别率,提高了人耳识别的鲁棒性,增强了人耳识别技术的实用性。     

Robust face recognition against expressions and partial occlusions

Facial features under variant-expressions and partial occlusions could have degrading effect on overall face recognition performance. As a solution, we suggest that the contribution of these features on final classification should be determined. In order to represent facial features contribution according to their variations, we propose a feature selection process that describes facial features as local independent component analysis(ICA) features. These local features are acquired using locally lateral subspace(LLS) strategy.Then, through linear discriminant analysis(LDA) we investigate the intraclass and interclass representation of each local ICA feature and express each feature s contribution via a weighting process. Using these weights, we define the contribution of each feature at local classifier level. In order to recognize faces under single sample constraint, we implement LLS strategy on locally linear embedding(LLE) along with the proposed feature selection. Additionally, we highlight the efficiency of the implementation of LLS strategy. The overall accuracy achieved by our approach on datasets with different facial expressions and partial occlusions such as AR, JAFFE,FERET and CK+ is 90.70%. We present together in this paper survey results on face recognition performance and physiological feature selection performed by human subjects.     

半监督类保持局部线性嵌入方法

为使局部线性嵌入（local linear embedding, LLE）这一无监督高维数据的非线性特征提取方法提取出的特征在分类或聚类学习上更优,提出一种半监督类保持局部线性嵌入(semi-supervised class preserving local linear embedding, SSCLLE)的非线性特征提取方法。该方法将半监督信息融入到LLE中,首先对标记样本近邻赋予伪标签,增大标记样本数量。其次,对标记样本之间的距离进行局部调整,缩小同类样本间距,扩大异类样本间距。同时在局部线性嵌入优化目标函数中增加全局同类样本间距和异类样本间距的约束项,使得提取出的低维特征可以确保同类样本点互相靠近,而异类样本点彼此分离。在一系列实验中,其聚类精确度以及可视化效果明显高于无监督LLE和现有半监督流特征提取方法,表明该方法提取出的特征具有很好的类保持特性。     

Locally Minimizing Embedding and Globally Maximizing Variance: Unsupervised Linear Difference Projection for Dimensionality Reduction

Recently, many dimensionality reduction algorithms, including local methods and global methods, have been presented. The representative local linear methods are locally linear embedding (LLE) and linear preserving projections (LPP), which seek to find an embedding space that preserves local information to explore the intrinsic characteristics of high dimensional data. However, both of them still fail to nicely deal with the sparsely sampled or noise contaminated datasets, where the local neighborhood structure is critically distorted. On the contrary, principal component analysis (PCA), the most frequently used global method, preserves the total variance by maximizing the trace of feature variance matrix. But PCA cannot preserve local information due to pursuing maximal variance. In order to integrate the locality and globality together and avoid the drawback in LLE and PCA, in this paper, inspired by the dimensionality reduction methods of LLE and PCA, we propose a new dimensionality reduction method for face recognition, namely, unsupervised linear difference projection (ULDP). This approach can be regarded as the integration of a local approach (LLE) and a global approach (PCA), so that it has better performance and robustness in applications. Experimental results on the ORL, YALE and AR face databases show the effectiveness of the proposed method on face recognition.     

改进重构权值的局部线性嵌入算法

目的 局部线性嵌入（LLE）算法是机器学习、数据挖掘等领域中的一种经典的流形学习算法。为克服LLE算法难以有效处理噪声、大曲率和稀疏采样数据等问题,提出一种改进重构权值的局部线性嵌入算法（IRWLLE）。方法 采用测地线距离来描述结构,重新构造和定义LLE中的重构权值,即在某样本的邻域内,将测地距离与欧氏距离之比定义为结构权值;将测地距离与中值测地距离之比定义为距离权值,再将结构权值与距离权值的乘积作为重构权值,从而将流形的结构和距离两种信息进行有机的结合。结果 对经典的人工数据Swiss roll、S-curve和Helix进行实验,在数据中加入噪声干扰,同时采用稀疏采样的方式来生成数据集,并与原始LLE算法和Hessian局部线性嵌入（HLLE）算法进行比较。实验结果表明,IRWLLE算法对比于LLE算法和HLLE算法,能够更好地保持流形的近邻关系,对流形的展开更加完好。尤其是对于加入噪声的大曲率数据集Helix,IRWLLE展现出极强的鲁棒性。对ORL和Yale人脸数据库进行人脸识别实验,采用最近邻分类器进行识别,将IRWLLE算法的识别结果与LLE算法进行对比。对于ORL数据集,IRWLLE算法识别率为90%,原LLE算法的识别率为85.5%;对于Yale数据集,IRWLLE算法识别率为88%,原LLE算法的识别率为75%,可见IRWLLE在人脸识别率上也有很大提高。结论 本文提出的IRWLLE算法对比于原LLE算法,不仅将流形距离信息引入到重构权值中,而且还将结构信息加入其中,有效减少了噪声和流形外数据点的干扰,所以对于噪声数据具有更强的鲁棒性,能够更好地处理稀疏采样数据和大曲率数据,在人脸识别率上也有较大提升。     

基于表情加权距离SLLE的人脸表情识别

局部线性嵌入(LLE)算法没有考虑训练样本的类别信息,而有监督LLE(SLLE)算法等同处理类别之间的差异性。根据人脸表情的特点,各个表情类别之间的差异性是有区别的,据此,文中构造一种基于表情加权距离的SLLE算法。在计算训练样本之间距离时,对来自不同表情类别的样本距离选择不同的加权值,从而使表情类别的先验信息得到更充分利用。在JAFFE库上进行人脸表情识别实验结果表明,相比LLE算法和SLLE算法,该算法在一定邻域范围内获得更好的人脸表情识别率,是一种有效算法。     

基于局部线性嵌套的虹膜识别方法

虹膜识别技术由于与其它生物特征识别技术相比具有更高的准确率而一直备受关注。本文采用局部线性嵌入算法对虹膜样本进行训练,来提取虹膜纹理特征,并以最小距离分类器作为决策空间的判决准则。实验表明本文提出的虹膜识别方法取得了良好的识别效果。     

基于局部线性嵌入的数字水印

提出了一种基于局部线性嵌入(LLE)的水印算法,它对仿射变换具有鲁棒性。该算法通过LLE内在的稳健性改善了鲁棒性。随机产生的水印被嵌入到局部线性嵌入的重建权的系数中。在提取水印时,水印几乎可以用像嵌入时一样的过程来提取。实验结果表明,该水印方案对仿射变换的鲁棒性较好。     

Supervised Discriminant Projection with Its Application to Face Recognition

In the past few decades, many face recognition methods have been developed. Among these methods, subspace analysis is an effective approach for face recognition. Unsupervised discriminant projection (UDP) finds an embedding subspace that preserves local structure information, and uncovers and separates embedding corresponding to different manifolds. Though UDP has been applied in many fields, it has limits to solve the classification tasks, such as the ignorance of the class information. Thus, a novel subspace method, called supervised discriminant projection (SDP), is proposed for face recognition in this paper. In our method, the class information was utilized in the procedure of feature extraction. In SDP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and class information. We test the performance of the proposed method SDP on three popular face image databases (i.e. AR database, Yale database, and a subset of FERET database). Experimental results show that the proposed method is effective.     

Dimensionality reduction-based spoken emotion recognition

To improve effectively the performance on spoken emotion recognition, it is needed to perform nonlinear dimensionality reduction for speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space. In this paper, a new supervised manifold learning algorithm for nonlinear dimensionality reduction, called modified supervised locally linear embedding algorithm (MSLLE) is proposed for spoken emotion recognition. MSLLE aims at enlarging the interclass distance while shrinking the intraclass distance in an effort to promote the discriminating power and generalization ability of low-dimensional embedded data representations. To compare the performance of MSLLE, not only three unsupervised dimensionality reduction methods, i.e., principal component analysis (PCA), locally linear embedding (LLE) and isometric mapping (Isomap), but also five supervised dimensionality reduction methods, i.e., linear discriminant analysis (LDA), supervised locally linear embedding (SLLE), local Fisher discriminant analysis (LFDA), neighborhood component analysis (NCA) and maximally collapsing metric learning (MCML), are used to perform dimensionality reduction on spoken emotion recognition tasks. Experimental results on two emotional speech databases, i.e. the spontaneous Chinese database and the acted Berlin database, confirm the validity and promising performance of the proposed method.     

基于LLE算法的人脸识别方法*

探讨了局部线性嵌入(LLE)算法的推导过程,提出了一种基于LLE算法的人脸识别方法,并实验分析了该方法在ORL和UMIST人脸数据库中的识别效果.     

Phoneme recognition using an adaptive supervised manifold learning algorithm

To effectively handle speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space, in this paper, an adaptive supervised manifold learning algorithm based on locally linear embedding (LLE) for nonlinear dimensionality reduction is proposed to extract the low-dimensional embedded data representations for phoneme recognition. The proposed method aims to make the interclass dissimilarity maximized, while the intraclass dissimilarity minimized in order to promote the discriminating power and generalization ability of the low-dimensional embedded data representations. The performance of the proposed method is compared with five well-known dimensionality reduction methods, i.e., principal component analysis, linear discriminant analysis, isometric mapping (Isomap), LLE as well as the original supervised LLE. Experimental results on three benchmarking speech databases, i.e., the Deterding database, the DARPA TIMIT database, and the ISOLET E-set database, demonstrate that the proposed method obtains promising performance on the phoneme recognition task, outperforming the other used methods.     

核方法与流形学习在心电识别中的研究

结合核方法和局部线性嵌入（LLE）方法,提出了一种基于核局部线性嵌入方法,该方法克服了局部线性嵌入方法由于心电特征分布不均衡造成的不稳定问题。结合支持向量机在MIT-BIH心律失常标准数据库进行实验,并利用PCA和LLE进行特征提取比较,验证了该方法的有效性及优势。     

模糊非相关判别转换及其应用

线性判别分析是一种特征提取和维数缩减的方法,广泛应用于人脸识别,语音识别和手写字母识别等领域。但是许多线性判别分析都是“硬”线性判别分析,每个数据点都严格地属于这一类或那一类。在非相关判别转换(UDT)基础上,提出了模糊非相关判别转换(FUDT)。FUDT是利用模糊集理论的有监督学习方法,其判别向量满足广义瑞利商方程,同时也满足样本到模糊非相关优化判别向量上的投影是非相关的。通过FUDT和UDT对公共数据库MSTAR的实验结果可看出,FUDT在处理SAR图像的特征提取方面优于UDT。     

Face recognition using nearest feature space embedding

Face recognition algorithms often have to solve problems such as facial pose, illumination, and expression (PIE). To reduce the impacts, many researchers have been trying to find the best discriminant transformation in eigenspaces, either linear or nonlinear, to obtain better recognition results. Various researchers have also designed novel matching algorithms to reduce the PIE effects. In this study, a nearest feature space embedding (called NFS embedding) algorithm is proposed for face recognition. The distance between a point and the nearest feature line (NFL) or the NFS is embedded in the transformation through the discriminant analysis. Three factors, including class separability, neighborhood structure preservation, and NFS measurement, were considered to find the most effective and discriminating transformation in eigenspaces. The proposed method was evaluated by several benchmark databases and compared with several state-of-the-art algorithms. According to the compared results, the proposed method outperformed the other algorithms.     

Orthogonal local spline discriminant projection with application to face recognition

In this paper, an efficient feature extraction algorithm called orthogonal local spline discriminant projection (O-LSDP) is proposed for face recognition. Derived from local spline embedding (LSE), O-LSDP not only inherits the advantages of LSE which uses local tangent space as a representation of the local geometry so as to preserve the local structure, but also makes full use of class information and orthogonal subspace to improve discriminant power. Extensive experiments on several standard face databases demonstrate the effectiveness of the proposed method.