基于分块非负矩阵分解人脸识别增量学习*

非负矩阵分解(NMF)算法可以提取图像的局部特征,然而NMF算法有两个主要缺点：a）当矩阵维数较大时,NMF算法非常耗时;b）当增加新的训练样本或类别时,NMF算法必须进行重复学习。为克服NMF算法这些缺点,提出了一种新的分块NMF算法(BNMF)。特别地,该方法还可用于增量学习。通过在FERET和CMU PIE人脸数据库上进行实验,结果表明该算法均优于NMF和PCA算法。     

Enhanced clustering of biomedical documents using ensemble non-negative matrix factorization

Searching and mining biomedical literature databases are common ways of generating scientific hypotheses by biomedical researchers. Clustering can assist researchers to form hypotheses by seeking valuable information from grouped documents effectively. Although a large number of clustering algorithms are available, this paper attempts to answer the question as to which algorithm is best suited to accurately cluster biomedical documents. Non-negative matrix factorization (NMF) has been widely applied to clustering general text documents. However, the clustering results are sensitive to the initial values of the parameters of NMF. In order to overcome this drawback, we present the ensemble NMF for clustering biomedical documents in this paper. The performance of ensemble NMF was evaluated on numerous datasets generated from the TREC Genomics track dataset. With respect to most datasets, the experimental results have demonstrated that the ensemble NMF significantly outperforms classical clustering algorithms of bisecting K-means, and hierarchical clustering. We compared four different methods for constructing an ensemble NMF. For clustering biomedical documents, this research is the first to compare ensemble NMF with typical classical clustering algorithms, and validates ensemble NMF constructed from different graph-based ensemble algorithms. This is also the first work on ensemble NMF with Hybrid Bipartite Graph Formulation for clustering biomedical documents.     

动态学习的非负矩阵分解算法

对现有增量型非负矩阵分解算法存在的一些缺陷进行改进,给出了一个基于误差判断的增量算法有效性准则.在此基础上,利用增加样本前的非负矩阵分解结果进行增量分解初始化,提出了一种新的动态非负矩阵分解算法.在多个数据集上的实验结果表明该算法可以实现对基矩阵和编码矩阵的即时更新,且具有较低的计算复杂度,在处理动态数据集时,还可有效识别噪声点,是一个有效的动态分解算法.     

稀疏约束图正则非负矩阵分解的增量学习算法

针对非负矩阵分解后数据的稀疏性降低、训练样本增多导致运算规模不断增大的现象,提出了一种稀疏约束图正则非负矩阵分解的增量学习算法。该方法不仅考虑数据的几何信息,而且对系数矩阵进行稀疏约束,并将它们与增量学习相结合。算法在稀疏约束和图正则化的条件下利用上一步的分解结果参与迭代运算,在节省大量运算时间的同时提高了分解后数据的稀疏性。在ORL和PIE人脸数据库上的实验结果表明了该算法的有效性。     

Structure preserving non-negative matrix factorization for dimensionality reduction

The problem of dimensionality reduction is to map data from high dimensional spaces to low dimensional spaces. In the process of dimensionality reduction, the data structure, which is helpful to discover the latent semantics and simultaneously respect the intrinsic geometric structure, should be preserved. In this paper, to discover a low-dimensional embedding space with the nature of structure preservation and basis compactness, we propose a novel dimensionality reduction algorithm, called Structure Preserving Non-negative Matrix Factorization (SPNMF). In SPNMF, three kinds of constraints, namely local affinity, distant repulsion, and embedding basis redundancy elimination, are incorporated into the NMF framework. SPNMF is formulated as an optimization problem and solved by an effective iterative multiplicative update algorithm. The convergence of the proposed update solutions is proved. Extensive experiments on both synthetic data and six real world data sets demonstrate the encouraging performance of the proposed algorithm in comparison to the state-of-the-art algorithms, especially some related works based on NMF. Moreover, the convergence of the proposed updating rules is experimentally validated.     

稀疏限制的增量式鲁棒非负矩阵分解及其应用

针对鲁棒非负矩阵分解（RNMF）的运算规模随训练样本数量逐渐增多而不断增大的问题，提出一种稀疏限制的增量式鲁棒非负矩阵分解算法。首先，对初始数据进行鲁棒非负矩阵分解；然后，将其分解结果参与到后续迭代运算；最后，在对系数矩阵增加稀疏限制的情况下与增量式学习相结合，使目标函数值在迭代求解时下降地更快。该算法在节省运算时间的同时提高了分解后数据的稀疏度。在数值实验中，将所提算法与鲁棒非负矩阵分解算法、稀疏限制的鲁棒非负矩阵分解（RNMFSC）算法进行了比较。在ORL和YALE人脸数据库上的实验结果表明，所提算法在运算时间和分解后数据的稀疏度等方面均优于其他两个算法，并且还具有较好的聚类效果，尤其在YALE人脸数据库上当聚类类别数为3时该算法的聚类准确率达到了91.67%。     

非负矩阵分解特征提取技术的研究进展

非负矩阵分解（non-negative matrix factorization,NMF）算法是在矩阵中所有元素均为非负的条件下对其实现的非负分解,基于非负矩阵分解的图像特征提取技术通过将图像表示为一系列非负基图像非减的叠加组合来提取图像的特征,这种特征提取方法不但具有良好的局部表征特性、有一定的稀疏性,而且对遮挡、光照不均及图像质量较差等情形具有卓越的效果。自正式提出以来,该方法得到了许多改进,但目前关于这些改进的综述都只是罗列了这些方法,并没有系统深入地分析,因而在大量阅读文献的基础上分析其内部联系,分类总结了非负矩阵分解的研究进展和各种改进方法的实质。首先介绍非负矩阵分解的基本思想,以手指静脉图像为例说明其应用于图像特征提取的方式,然后重点深入讨论了非负矩阵分解方法的改进算法,提出了非负矩阵分解应用中有待进一步研究的新问题。     

Constrained non-negative matrix factorization for score-informed piano music restoration

In this work, we propose a constrained non-negative matrix factorization method for the audio restoration of piano music using information from the score. In the first stage (instrument training), spectral patterns for the target source (piano) are learned from a dataset of isolated piano notes. The model for the piano is constrained to be harmonic because, in this way, each pattern can define a single pitch. In the second stage (noise training), spectral patterns for the undesired source (noise) are learned from the most common types of vinyl noises. To obtain a representative model for the vinyl noise, a cross-correlation-based constraint that minimizes the cross-talk between different noise components is used. In the final stage (separation), we use the trained instrument and noise models in an NMF framework to extract the clean audio signal from undesired non-stationary noise. To improve the separation results, we propose a novel score-based constraint to avoid activations of notes or combinations that are not present in the original score. The proposed approach has been evaluated and compared with commercial audio restoration softwares, obtaining competitive results.     

基于非负矩阵分解与稀疏表示的多标签分类算法

传统的多标签分类算法是以二值标签预测为基础的,而二值标签由于仅能指示数据是否具有相关类别,所含语义信息较少,无法充分表示标签语义信息。为充分挖掘标签空间的语义信息,提出了一种基于非负矩阵分解和稀疏表示的多标签分类算法（MLNS）。该算法结合非负矩阵分解与稀疏表示技术,将数据的二值标签转化为实值标签,从而丰富标签语义信息并提升分类效果。首先,对标签空间进行非负矩阵分解以获得标签潜在语义空间,并将标签潜在语义空间与原始特征空间结合以形成新的特征空间;然后,对此特征空间进行稀疏编码来获得样本间的全局相似关系;最后,利用该相似关系重构二值标签向量,从而实现二值标签与实值标签的转化。在5个标准多标签数据集和5个评价指标上将所提算法与MLBGM、ML²、LIFT和MLRWKNN等算法进行对比。实验结果表明,所提MLNS在多标签分类中优于对比的多标签分类算法,在50%的案例中排名第一,在76%的案例中排名前二,在全部的案例中排名前三。     

基于图学习正则判别非负矩阵分解的人脸识别

基于图正则非负矩阵分解（NMF）算法充分利用了高维数据通常位于一个低维流形空间的假设从而构造拉普拉斯矩阵,但该算法的缺点是构造出的拉普拉斯矩阵是提前计算得到的,并没有在乘性更新过程中对它进行迭代。为了解决这个问题,结合子空间学习中的自表示方法生成表示系数,并进一步计算相似性矩阵从而得到拉普拉斯矩阵,而且在更新过程中对拉普拉斯矩阵进行迭代。另外,利用训练集的标签信息构造类别指示矩阵,并引入两个不同的正则项分别对该类别指示矩阵进行重构。该算法被称为图学习正则判别非负矩阵分解（GLDNMF）,并给出了相应的乘性更新规则和目标函数的收敛性证明。在两个标准数据集上的人脸识别实验结果显示,和现有典型算法相比,所提算法的人脸识别的准确率提升了1% ~ 5%,验证了其有效性。     

融合矩阵补全与深度矩阵分解的推荐算法

深度矩阵分解采用深层非线性映射,从而突破了矩阵分解中双线性关系影响推荐系统性能的瓶颈,但它没有考虑用户对未评分项目的偏好,且对于稀疏性较高的大规模数据其推荐性能不具有优势,为此提出一种融合矩阵补全与深度矩阵分解的推荐算法.首先通过矩阵补全模型将原始评分矩阵中的未知元素进行填补,然后依据补全后的矩阵,利用深度学习模型分别构建用户和项目潜在向量.最后,在MovieLens和SUSHI数据集上进行测试,实验结果表明,与深度矩阵分解相比,所提算法显著地提高了推荐系统的性能.     

基于图正则化的半监督非负矩阵分解

提出了一种基于图正则化的半监督非负矩阵分解算法（GSNMF）,克服了非负矩阵分解（NMF）、约束非负矩阵分解（CNMF）和图正则化非负矩阵分解（GNMF）方法忽略样本数据的局部几何结构或标签信息不足的缺陷,且NMF、CNMF和GNMF均为GSNMF的特例。也从理论上证明了GSNMF算法的收敛性。该算法对样本数据进行低维非负分解时,在图框架下既保持数据的几何结构,又利用已知样本的标签信息,在进行半监督学习时,同类样本能更好地聚集而类间距离尽可能大。在人脸数据库ORL、FERET和手写体数据库USPS上的仿真结果表明,相对于NMF及其一些改进算法,GSNMF均具有更高的聚类精度。     

稀疏约束下非负矩阵分解的增量学习算法

非负矩阵分解(NMF)是一种有效的子空间降维方法。为了改善非负矩阵分解运算规模随训练样本增多而不断增大的现象,同时提高分解后数据的稀疏性,提出了一种稀疏约束下非负矩阵分解的增量学习算法,该算法在稀疏约束的条件下利用前一次分解的结果参与迭代运算,在节省大量运算时间的同时提高了分解后数据的稀疏性。在ORL和CBCL人脸数据库上的实验表明了该算法降维的有效性。     

Non-negative matrix factorization for semi-supervised data clustering

Traditional clustering algorithms are inapplicable to many real-world problems where limited knowledge from domain experts is available. Incorporating the domain knowledge can guide a clustering algorithm, consequently improving the quality of clustering. In this paper, we propose SS-NMF: a semi-supervised non-negative matrix factorization framework for data clustering. In SS-NMF, users are able to provide supervision for clustering in terms of pairwise constraints on a few data objects specifying whether they “must” or “cannot” be clustered together. Through an iterative algorithm, we perform symmetric tri-factorization of the data similarity matrix to infer the clusters. Theoretically, we show the correctness and convergence of SS-NMF. Moveover, we show that SS-NMF provides a general framework for semi-supervised clustering. Existing approaches can be considered as special cases of it. Through extensive experiments conducted on publicly available datasets, we demonstrate the superior performance of SS-NMF for clustering.

非负矩阵分解降维的入侵检测方法

直接对高维网络连接数据进行处理会出现维数灾难问题,因此,需要对其进行维数约简。非负矩阵分解不仅能对高维数据进行降维,而且使矩阵在分解后的所有分量均为非负值,符合网络连接数据的语义特征。将其应用到入侵检测中,把高维数据投影到低维可视空间上,用散点来表示网络连接记录,通过观察散点所处位置来判断其所属类别,实现入侵检测的可视化。实验验证了这种入侵检测方法的有效性。     

一种基于非负矩阵分解的鲁棒零水印算法

为了解决现有数字水印中鲁棒性和不可感知性之间的矛盾,设计了一种基于非负矩阵分解和离散小波变换的图像零水印算法。原始图像进行不重叠分块,分别对每子块图像进行3级小波分解得到低频近似分量;对细节分量作非负矩阵分解得到可近似表示子块图像的基矩阵和系数矩阵;将系数矩阵量化得到特征向量,通过特征向量和水印的运算得到原始图像的版权信息。实验结果表明该方案对常见信号处理具有很强的鲁棒性,同时密钥的使用保障了算法的安全性。     

图正则化稀疏判别非负矩阵分解

非负矩阵分解是一种流行的数据表示方法,利用图正则化约束能有效地揭示数据之间的局部流形结构。为了更好地提取图像特征,给出了一种基于图正则化的稀疏判别非负矩阵分解算法（graph regularization sparse discriminant non-negative matrix factorization,GSDNMF-L_2,1）。利用同类样本之间的稀疏线性表示来构建对应的图及权矩阵;以L_2,1范数进行稀疏性约束;以最大间距准则为优化目标函数,利用数据集的标签信息来保持数据样本之间的流形结构和特征的判别性,并给出了算法的迭代更新规则。在若干图像数据集上的实验表明,GSDNMF-L_2,1在特征提取方面的分类精度优于各对比算法。     

Subspace learning for unsupervised feature selection via matrix factorization

Dimensionality reduction is an important and challenging task in machine learning and data mining. Feature selection and feature extraction are two commonly used techniques for decreasing dimensionality of the data and increasing efficiency of learning algorithms. Specifically, feature selection realized in the absence of class labels, namely unsupervised feature selection, is challenging and interesting. In this paper, we propose a new unsupervised feature selection criterion developed from the viewpoint of subspace learning, which is treated as a matrix factorization problem. The advantages of this work are four-fold. First, dwelling on the technique of matrix factorization, a unified framework is established for feature selection, feature extraction and clustering. Second, an iterative update algorithm is provided via matrix factorization, which is an efficient technique to deal with high-dimensional data. Third, an effective method for feature selection with numeric data is put forward, instead of drawing support from the discretization process. Fourth, this new criterion provides a sound foundation for embedding kernel tricks into feature selection. With this regard, an algorithm based on kernel methods is also proposed. The algorithms are compared with four state-of-the-art feature selection methods using six publicly available datasets. Experimental results demonstrate that in terms of clustering results, the proposed two algorithms come with better performance than the others for almost all datasets we experimented with here.     

丰富图像标签的正则化非负矩阵分解方法

目的 随着Web2.0技术的进步,以用户生成内容为中心的社交网站蓬勃发展,也使得基于图像标签的图像检索技术越来越重要。但是,由于用户标注时的随意性和个性化,导致用户提交的图像标签不够完备,降低了图像检索的准确性。方法 针对这一问题,提出一种正则化的非负矩阵分解方法来丰富图像欠完备的标签,提高图像标签的完备性。利用非负矩阵分解的方法将原始的标签-图像矩阵投影到潜在的低秩空间里消除噪声,同时利用图像的类内视觉离散度作为正则化项提高消除噪声、丰富标签的效果。结果 利用从社交网站Flickr上下载的大量社交图像进行对比实验,验证了本文方法对丰富图像标签的有效性。通过对比目前流行的优化算法,本文算法获得较高的性能提升,算法平均准确度提高了12.3%。结论 将图像类内视觉离散度作为正则化项的非负矩阵分解算法,能较好地丰富社交图像的标签,解决网络图像标签的欠完备问题。