Discriminant Nonnegative Tensor Factorization Algorithms

Nonnegative matrix factorization (NMF) has proven to be very successful for image analysis, especially for object representation and recognition. NMF requires the object tensor (with valence more than one) to be vectorized. This procedure may result in information loss since the local object structure is lost due to vectorization. Recently, in order to remedy this disadvantage of NMF methods, nonnegative tensor factorizations (NTF) algorithms that can be applied directly to the tensor representation of object collections have been introduced. In this paper, we propose a series of unsupervised and supervised NTF methods. That is, we extend several NMF methods using arbitrary valence tensors. Moreover, by incorporating discriminant constraints inside the NTF decompositions, we present a series of discriminant NTF methods. The proposed approaches are tested for face verification and facial expression recognition, where it is shown that they outperform other popular subspace approaches.      

基于全局不相关的多流形学习

为提升人脸识别算法的鲁棒性,减少判别信息的冗余度,提出基于全局不相关的多流形判别学习算法(UFDML)。使用特征空间到特征空间的距离,学习样本局部判别信息,提出全局不相关约束,使提取的判别特征是统计不相关的。在Yale,AR,ORL人脸库上的实验结果表明,与LPP(局部保持投影)、LDA(线性判别分析)、UDP(非监督判别投影)等人脸识别算法相比,所提算法的平均识别率高于其它算法,验证了其有效性。     

Local sparse representation projections for face recognition

How to define the sparse affinity weight matrices is still an open problem in existing manifold learning algorithm. In this paper, we propose a novel supervised learning method called local sparse representation projections (LSRP) for linear dimensionality reduction. Differing from sparsity preserving projections (SPP) and the recent manifold learning methods such as locality preserving projections (LPP), LSRP introduces the local sparse representation information into the objective function. Although there are no labels used in the local sparse representation, it still can provide better measure coefficients and significant discriminant abilities. By combining the local interclass neighborhood relationships and sparse representation information, LSRP aims to preserve the local sparse reconstructive relationships of the data and simultaneously maximize the interclass separability. Comprehensive comparison and extensive experiments show that LSRP achieves higher recognition rates than principle component analysis, linear discriminant analysis and the state-of-the-art techniques such as LPP, SPP and maximum variance projections.     

新的局部保持鉴别分析算法

提出了一种新的局部保持鉴别分析算法：基于迹比准则与自适应近邻图嵌入的局部保持鉴别分析算法。根据样本分布特性自适应构建类内和类间近邻图,保持数据的局部结构并且利用数据的鉴别信息,定义局部类内离差矩阵以及局部类间离差矩阵,采用迹比Fisher判别函数作为目标函数,通过迭代的方法最大化局部类间离差矩阵与类内离差矩阵的迹比值,解得最优子空间。在ORL和Yale人脸数据库上的实验表明该方法是有效的。     

图正则化稀疏判别非负矩阵分解

非负矩阵分解是一种流行的数据表示方法,利用图正则化约束能有效地揭示数据之间的局部流形结构。为了更好地提取图像特征,给出了一种基于图正则化的稀疏判别非负矩阵分解算法（graph regularization sparse discriminant non-negative matrix factorization,GSDNMF-L_2,1）。利用同类样本之间的稀疏线性表示来构建对应的图及权矩阵;以L_2,1范数进行稀疏性约束;以最大间距准则为优化目标函数,利用数据集的标签信息来保持数据样本之间的流形结构和特征的判别性,并给出了算法的迭代更新规则。在若干图像数据集上的实验表明,GSDNMF-L_2,1在特征提取方面的分类精度优于各对比算法。     

Face recognition using second-order discriminant tensor subspace analysis

Discriminant information (DI) plays a critical role in face recognition. In this paper, we proposed a second-order discriminant tensor subspace analysis (DTSA) algorithm to extract discriminant features from the intrinsic manifold structure of the tensor data. DTSA combines the advantages of previous methods with DI, the tensor methods preserving the spatial structure information of the original image matrices, and the manifold methods preserving the local structure of the samples distribution. DTSA defines two similarity matrices, namely within-class similarity matrix and between-class similarity matrix. The within-class similarity matrix is determined by the distances of point pairs in the same class, while the between-class similarity matrix is determined by the distances between the means of each pair of classes. Using these two matrices, the proposed method preserves the local structure of the samples to fit the manifold structure of facial images in high dimensional space better than other methods. Moreover, compared to the 2D methods, the tensor based method employs two-sided transformations rather than single-sided one, and yields higher compression ratio. As a tensor method, DTSA uses an iterative procedure to calculate the optimal solution of two transformation matrices. In this paper, we analyzed DTSA's connections to 2D-DLPP and TSA, theoretically. The experiments on the ORL, Yale and YaleB facial databases show the effectiveness of the proposed method.     

Image representation using Laplacian regularized nonnegative tensor factorization

Tensor provides a better representation for image space by avoiding information loss in vectorization. Nonnegative tensor factorization (NTF), whose objective is to express an n-way tensor as a sum of k rank-1 tensors under nonnegative constraints, has recently attracted a lot of attentions for its efficient and meaningful representation. However, NTF only sees Euclidean structures in data space and is not optimized for image representation as image space is believed to be a sub-manifold embedded in high-dimensional ambient space. To avoid the limitation of NTF, we propose a novel Laplacian regularized nonnegative tensor factorization (LRNTF) method for image representation and clustering in this paper. In LRNTF, the image space is represented as a 3-way tensor and we explicitly consider the manifold structure of the image space in factorization. That is, two data points that are close to each other in the intrinsic geometry of image space shall also be close to each other under the factorized basis. To evaluate the performance of LRNTF in image representation and clustering, we compare our algorithm with NMF, NTF, NCut and GNMF methods on three standard image databases. Experimental results demonstrate that LRNTF achieves better image clustering performance, while being more insensitive to noise.     

基于改进的非负矩阵分解的人脸表情识别

针对存在部分遮挡的人脸,提出了一种基于改进的非负矩阵分解的人脸表情识别方法,首先,用改进的非负矩阵分解算法对人脸图像进行表情特征提取,然后用最大相关分类器对面部表情进行分类。在Cohn-Kanade人脸表情数据库上的实验,结果表明,该方法提高了无遮挡的人脸表情识别,对有遮挡的人脸表情识别也有改善。     

基于图正则化和稀疏约束的半监督非负矩阵分解

非负矩阵分解是在矩阵非负约束下的分解算法。为了提高识别率,提出了一种基于稀疏约束和图正则化的半监督非负矩阵分解方法。该方法对样本数据进行低维非负分解时,既保持数据的几何结构,又利用已知样本的标签信息进行半监督学习,而且对基矩阵施加稀疏性约束,最后将它们整合于单个目标函数中。构造了一个有效的更新算法,并且在理论上证明了该算法的收敛性。在多个人脸数据库上的仿真结果表明,相对于NMF、GNMF、CNMF等算法,GCNMFS具有更好的聚类精度和稀疏性。     

组合2DFLDA监督的非负矩阵分解和独立分量分析的特征提取方法*

通过对投影非负矩阵分解(NMF)和二维Fisher线性判别的分析,针对NMF的特征提取存在无监督学习以及特征维数高的问题,提出了组合2DFLDA监督的非负矩阵分解和独立分量分析(SPGNMFICA)的特征提取方法。首先对样本进行投影梯度的非负矩阵分解,将得到的NMF子图像进行二维Fisher线性判别,主要反映类间差异信息构建子空间;对子空间的向量进行独立分量分析(ICA),得到独立分量特征空间;其次将样本在独立分量特征空间上进行投影;最后使用径向基网络对投影系数进行识别。通用人脸库ORL和YALE的识别实验证明,该算法是一种有效的特征提取和识别方法。     

Total Variation Constrained Non-Negative Matrix Factorization for Medical Image Registration

This paper presents a novel medical image registration algorithm named total variation constrained graphregularization for non-negative matrix factorization(TV-GNMF).The method utilizes non-negative matrix factorization by total variation constraint and graph regularization.The main contributions of our work are the following.First,total variation is incorporated into NMF to control the diffusion speed.The purpose is to denoise in smooth regions and preserve features or details of the data in edge regions by using a diffusion coefficient based on gradient information.Second,we add graph regularization into NMF to reveal intrinsic geometry and structure information of features to enhance the discrimination power.Third,the multiplicative update rules and proof of convergence of the TV-GNMF algorithm are given.Experiments conducted on datasets show that the proposed TV-GNMF method outperforms other state-of-the-art algorithms.     

Robust non-negative matrix factorization via joint sparse and graph regularization for transfer learning

In real-world applications, we often have to deal with some high-dimensional, sparse, noisy, and non-independent identically distributed data. In this paper, we aim to handle this kind of complex data in a transfer learning framework, and propose a robust non-negative matrix factorization via joint sparse and graph regularization model for transfer learning. First, we employ robust non-negative matrix factorization via sparse regularization model (RSNMF) to handle source domain data and then learn a meaningful matrix, which contains much common information between source domain and target domain data. Second, we treat this learned matrix as a bridge and transfer it to target domain. Target domain data are reconstructed by our robust non-negative matrix factorization via joint sparse and graph regularization model (RSGNMF). Third, we employ feature selection technique on new sparse represented target data. Fourth, we provide novel efficient iterative algorithms for RSNMF model and RSGNMF model and also give rigorous convergence and correctness analysis separately. Finally, experimental results on both text and image data sets demonstrate that our REGTL model outperforms existing start-of-art methods.     

NMF与LDA相结合的彩色人脸识别

为了提高彩色人脸识别的性能,提出了一种非负矩阵分解与线性判别分析相结合的彩色人脸识别算法。首先采用非负矩阵分解算法对彩色人脸图像不同颜色通道的信息进行编码,计算彩色人脸图像空间的基图像;然后根据非负矩阵分解计算得到的图像分解系数,融入人脸对象的类别信息,采用线性判别分析算法计算最优的鉴别子空间;最后以彩色人脸图像的投影系数为特征,采用最近邻分类算法进行人脸识别。在CVL和CMUPIE人脸数据库上的实验结果验证了提出的彩色人脸识别算法的正确性和有效性。     

稀疏约束图正则非负矩阵分解

非负矩阵分解(NMF)是在矩阵非负约束下的一种局部特征提取算法。为了提高识别率,提出了稀疏约束图正则非负矩阵分解方法。该方法不仅考虑数据的几何信息,而且对系数矩阵进行稀疏约束,并将它们整合于单个目标函数中。构造了一个有效的乘积更新算法,并且在理论上证明了该算法的收敛性。在ORL和MIT-CBCL人脸数据库上的实验表明了该算法的有效性。     

Batch process monitoring with tensor factorization

A batch process monitoring method using tensor factorization, tensor locality preserving projections (TLPP), is proposed. In many existing vector-based methods on batch process monitoring such as MPCA and MLPP, a batch data is represented as a vector in high-dimensional space. But vectorizing batch data will lead to information loss. Essentially, a batch data is presented as a second order tensor, or a matrix. In this case, tensor factorization may be used to deal with the two-way batch data matrix directly instead of performing vectorizing procedure. Furthermore, tensor representation has some advantages such as low memory and storage requirements and less estimated parameters for normal operating condition (NOC) model. On the other hand, different from principal component analysis (PCA) which aims at preserving the global Euclidean structure of the data, the TLPP aims to preserve the local neighborhood information and to detect the intrinsic manifold structure of the data. Consequently, TLPP may be used to find more meaningful intrinsic information hidden in the observations. The effectiveness and advantages of the TLPP monitoring approach are tested with the data from a benchmark fed-batch penicillin fermentation and two industrial fermentation processes, penicillin and cephalosporin, respectively.     

Graph dual regularization non-negative matrix factorization for co-clustering

Low-rank matrix factorization is one of the most useful tools in scientific computing, data mining and computer vision. Among of its techniques, non-negative matrix factorization (NMF) has received considerable attention due to producing a parts-based representation of the data. Recent research has shown that not only the observed data are found to lie on a nonlinear low dimensional manifold, namely data manifold, but also the features lie on a manifold, namely feature manifold. In this paper, we propose a novel algorithm, called graph dual regularization non-negative matrix factorization (DNMF), which simultaneously considers the geometric structures of both the data manifold and the feature manifold. We also present a graph dual regularization non-negative matrix tri-factorization algorithm (DNMTF) as an extension of DNMF. Moreover, we develop two iterative updating optimization schemes for DNMF and DNMTF, respectively, and provide the convergence proofs of our two optimization schemes. Experimental results on UCI benchmark data sets, several image data sets and a radar HRRP data set demonstrate the effectiveness of both DNMF and DNMTF.     

Tensor rank one differential graph preserving analysis for facial expression recognition

This paper presents a new dimensionality reduction algorithm for multi-dimensional data based on the tensor rank-one decomposition and graph preserving criterion. Through finding proper rank-one tensors, the algorithm effectively enhances the pairwise inter-class margins and meanwhile preserves the intra-class local manifold structure. In the algorithm, a novel marginal neighboring graph is devised to describe the pairwise inter-class boundaries, and a differential formed objective function is adopted to ensure convergence. Furthermore, the algorithm has less computation in comparison with the vector representation based and the tensor-to-tensor projection based algorithms. The experiments for the basic facial expressions recognition show its effectiveness, especially when it is followed by a neural network classifier.     

复杂电磁环境下通信辐射源个体细微特征提取方法

针对实际复杂电磁环境下通信辐射源个体细微特征提取面临的标签样本缺失问题,将半监督学习理论引入到通信辐射源细微特征提取,提出一种半监督框架下的局部近邻保持正则化判别分析方法。该方法在双谱估计的基础上,通过向线性判别模型中有效融入由无标签样本所提供的流形结构信息,从而将线性判别方法扩展到半监督学习。在实际采集的同种型号、同种厂家、相同批次以及相同工作模式的不同FM通信电台数据集上的实验结果表明,该方法能够获得更优的分类识别性能。     

基于特征子空间邻域的局部保持流形学习算法*

局部保持流形学习算法通过保持局部邻域特性来挖掘隐藏在高维数据中的内在流形结构。然而,对于缺乏足够训练样本的高维数据集,或者高维数据集存在非线性结构和高维数据特征中存在冗余、干扰特征,使得在原特征空间中利用欧式距离定义的邻域关系并不能真实反映数据的内在流形结构,从而影响算法的性能。提出利用正约束寻找特征子空间的方法,使得在此子空间中更多的同类样本紧聚,并进一步在该子空间中构建邻域关系来挖掘高维数据的内在流形,形成基于特征子空间邻域特性的局部保持流形学习算法(NFS-LPP和NFS-NPE)。它们在一定程度上克服了高维小样本数据集难以正确挖掘内在流形结构的问题,在Yale和ORL人脸库上的分类和聚类实验验证了其有效性。     

Facial expression recognition based on two-dimensional discriminant locality preserving projections

In this paper, a novel method called two-dimensional discriminant locality preserving projections (2D-DLPP) is proposed. By introducing between-class scatter constraint and label information into two-dimensional locality preserving projections (2D-LPP) algorithm, 2D-DLPP successfully finds the subspace which can best discriminate different pattern classes. So the subspace obtained by 2D-DLPP has more discriminant power than 2D-LPP, and is more suitable for recognition tasks. The proposed method was applied to facial expression recognition tasks on JAFFE and Cohn-Kanade database and compared with other three widely used two-dimensional methods: 2D-PCA, 2D-LDA and 2D-LPP. The high recognition rates show the effectiveness of the proposed algorithm.